JP2001282279A - Voice information processing method and apparatus, and storage medium - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To enable a duration of a phoneme sequence to be set with high accuracy, and to give a natural phoneme time according to a phoneme / language environment. SOLUTION: Based on a duration model of a global segment, a duration of a phoneme sequence of a predetermined unit is obtained (S302). Based on the duration model of the local segment, the duration of each phoneme constituting the phoneme sequence is determined (S303). The duration of each phoneme is set based on the duration of the phoneme series and the duration of each phoneme (S304).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech information processing method and apparatus for setting a duration of a phoneme performed in speech synthesis, and a computer-readable program storing a program for executing the speech synthesis method. It concerns a possible storage medium.

[0002]

2. Description of the Related Art In recent years, a speech synthesizer has been developed which converts an arbitrary character sequence into a phoneme sequence and converts the phoneme sequence into a synthesized speech according to a predetermined speech rule synthesis method.

[0003]

The synthesized speech output from the conventional speech synthesizer is unnatural and mechanical as compared to natural speech uttered by humans.

As one of the causes, for example, a phoneme sequence “o, X, s,
In “e, i”, the accuracy of the control rule of the phoneme duration for generating the duration of each phoneme can be mentioned. If the accuracy is low, the duration is not properly given to each phoneme, so that the synthesized speech is unnatural and mechanical.

SUMMARY OF THE INVENTION The present invention has been made in view of the above conventional example, and is capable of accurately setting the duration of a phoneme sequence and providing a natural phoneme time length corresponding to a phoneme / language environment. It is an object to provide a method and an apparatus thereof.

[0006]

In order to achieve the above object, a voice information processing apparatus according to the present invention has the following arrangement. In other words, based on the duration model of the global segment, means for determining the duration of the phoneme sequence in a predetermined unit, and based on the duration model of the local segment,
Means for determining the duration of each phoneme constituting the phoneme sequence, and setting means for setting the duration of each phoneme based on the duration of the phoneme sequence and the duration of each phoneme. Speech synthesis means for synthesizing speech based on the duration of each phoneme set by the setting means.

[0007] To achieve the above object, a voice information processing method of the present invention comprises the following steps. That is, based on the duration model of the global segment, a step of obtaining the duration of the phoneme sequence of a predetermined unit, and, based on the duration model of the local segment, the continuation of each phoneme constituting the phoneme sequence. Determining the time length, based on the duration of the phoneme sequence and the duration of each phoneme,
A setting step of setting a duration time of each of the phonemes; and a speech synthesis step of synthesizing speech based on the duration time of each of the phonemes set in the setting step.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

[First Embodiment] FIG. 1 is a block diagram showing a configuration of a speech synthesizer according to a first embodiment of the present invention.

In FIG. 1, reference numeral 101 denotes a CPU and a ROM
According to the control program stored in the RAM 102 or the control program loaded into the RAM 103 from the external storage device 104, various controls in the speech synthesizer of the present embodiment are performed. ROM 102 stores various parameters and CPU
The control program 101 executes a control program.
The RAM 103 provides a work area when the CPU 101 executes various controls, and stores a control program executed by the CPU 101. 104 is a hard disk, a floppy (registered trademark) disk, a CD-RO
When the external storage device is a hard disk, various programs installed from a CD-ROM, a floppy disk or the like are stored. An input unit 105 has a pointing device such as a keyboard and a mouse. Also, the input unit 105
May input data from the Internet or the like via a communication line or the like. A display unit 106 such as a liquid crystal display or a CRT displays various data under the control of the CPU 101. A speaker 107 converts an audio signal (electric signal) into an audible sound and outputs the sound. Reference numeral 108 denotes a bus that connects the above components. Reference numeral 109 denotes a speech synthesis unit.

FIG. 2 is a flowchart showing the operation of the speech synthesis unit 109 according to the first embodiment. Each step described below is performed by using a control program stored in the ROM 102 or the RAM 1 from the external storage device 104.
This is realized by the CPU 101 executing the control program loaded in the CPU 03.

First, in step S201, when Japanese text data mixed with kanji and kana is input from the input unit 105, the process proceeds to step S202, where the input text data is analyzed using the linguistic analysis dictionary 201, and the input text is input. It extracts information such as phonemic sequences (reading) and accents for the data. Next, the process proceeds to step S203, and a prosody (referred to as prosodic information) such as a duration time, a fundamental frequency (pitch pattern), and power of each phoneme constituting the phoneme sequence obtained in step S202 is generated using the information. . At this time, the duration of the phoneme is determined using the duration model 202, and the fundamental frequency, power, and the like are determined using the prosody control model 203.

Next, proceeding to step S204,
Phonemic sequence analyzed and extracted in step 202, and step S
Based on the prosody generated in 203, a plurality of speech units (waveforms or characteristic parameters) for generating a synthesized speech corresponding to the phoneme sequence are selected from the speech unit dictionary 204. Next, the process proceeds to step S205, in which a synthesized voice signal is generated using the selected voice segments, and in step S206, a voice is output from the speaker 107 based on the generated synthesized voice signal. Finally, in step S207, it is determined whether or not all the processes for the input text data have been completed. If not, the process returns to step S201 to continue the above-described processes.

FIG. 3 is a flowchart for explaining in detail a part of the prosody generation processing in step S203 in FIG. In FIG. 3, using the continuous time length model 202,
A procedure for setting the duration of a phoneme sequence of a predetermined unit (hereinafter, referred to as a global segment) and the duration of each phoneme (hereinafter, referred to as a local segment) constituting the phoneme sequence will be described. Here, the duration model 202 is
A duration model for a global segment (also referred to as a global duration model) 301 and a duration model for a local segment (also referred to as a local duration model) 302 are included.

First, in step S301, an analysis result for input text data obtained by the text processing in step S202 of FIG. 2 is input. Here, as the analysis result, there is a phoneme environment obtained from phoneme information such as phonemes, information on a language environment obtained from language information such as the number of mora, the number of accent phrases, and the part of speech. Next, the process proceeds to step S302, where the duration of the global segment is set based on the global duration model 301 for the global segment. Here, the global segment is composed of utterances such as accent phrases, words, phrases, sentences, etc., which can be processed as a unit (called an utterance unit).

Next, the process proceeds to step S303, where the duration of the local segment is set based on the local duration model 302 for the local segment. Here, the local segment is composed of phoneme units constituting speech units such as phonemes, syllables, and mora.

Finally, the process proceeds to step S304, where the duration of the global segment obtained by the sum of the durations of the local segments obtained in step S303 and the duration of the global segment set in step S302 are determined. The difference with the duration is determined by the step S
The local duration of each phoneme is determined by expanding and contracting the duration of a local segment using the local duration expansion / contraction model 303 so as to have the global duration set at 302. .

As a specific example, when "flower" is input as text data, a phoneme sequence analyzed from individual character strings is set as a global segment, and this is divided into local segments using mora as a phoneme unit. Then "ha""na"
It becomes "ga". Where the average duration of each mora (durat
ion) is, for example, 100 milliseconds, and assuming that the actual measured time length of the global segment is 600 milliseconds, the time length of the global segment is 600 milliseconds, whereas the local segment has a time length of 600 milliseconds. The overall duration obtained from the sum of the durations is 300 milliseconds,
There will be a millisecond difference.

Next, the method of creating the global duration model 301 for the global segment and the process of setting the duration for the global segment in step S302 will be described with reference to the flowchart of FIG. .

FIG. 4 is a flowchart showing a method of creating a global duration model 301 for a global segment.

First, in step S401, an audio file 40 having a plurality of learning samples for creating a global duration model for a global segment.
1 and a side information file 4 having information necessary for extracting a duration length such as start and end time information of phonemes and syllables.
02 to extract the global duration. Next, proceeding to step S402, a phoneme / language environment file 403 having information on a language environment obtained from phonetic information such as phonemes and linguistic information obtained from linguistic information such as the number of mora, the number of accent phrases, and part of speech, and a step S401. Using the extracted global duration information, a global duration model 301 that takes into account a predetermined language environment is created.

The specific processing procedure is as follows. Let K be the number of learning samples in the audio file 401 for creating the duration model 301 of the global segment,
Let dk be the duration of the global segment in the k-th learning sample. In the present embodiment, instead of creating a model for directly predicting the global duration d k, the global segment average duration ~ d obtained from K learning samples is used to calculate the global segment. A model for predicting sk, in which the duration dk is normalized by sk = dk / kd (1), is created. Here, the average duration length d of the global segment can be obtained by various methods. For example, when dk is the average mora duration length (average duration length per mora), = (1 / K) Σ (dk / Nk) (Σ is the sum of k = 1 to K) ... It can be obtained as equation (2). Here, Nk is the number of moras in the k-th learning sample.

At this time, the prediction value sk of sk, which is obtained by normalizing the global duration dk, can be obtained by the following equation using a linear multiple regression analysis method.

^ Sk = a0 + ΣΣai, j × xk, i, j (the first Σ indicates the sum of i = 1 to I, and the next Σ indicates the sum of j = 1 to Ji) Expression (3) where I Is the number of phonemes / language environment factors (items), Ji
Represents the number of categories for the factor i (for example, phoneme type or number of accent phrases). Xk, i, j is an explanatory variable in category j (for example, phoneme set or accent type) of factor i of sample k, ai, j is a regression coefficient for category j of factor i, and a0 is a constant term. . Using the predicted value ^ sk, the global duration length dk of the global segment for the k-th sample is calculated from Expression (1) as follows: ^ dk = ^ sk x ~ d ... Expression (4) Can be. This equation (4) becomes the global time length model 301.

The values of I and Ji can be selected in various ways. For example, the phoneme type and the number of accent phrases in the global segment are selected as the factor i, and 26 types of phoneme sets and global When the number of accent phrases (1, 2, 3, 4 or more) in the target segment is selected, I = 2, J1 = 26, and J2 = 4.

Next, a method of creating a local duration model 302 for a local segment, and step S30
The process of setting the local duration for the three local segments will be described with reference to the flowchart in FIG. These processes are performed as follows in the same manner as in the global segment.

FIG. 5 is a flowchart showing a method for creating a local duration model 302 for a local segment.

First, in step S501, an audio file 501 having a plurality of learning samples for creating a duration model for a local segment;
A side information file 5 having information necessary for extracting a duration such as start and end time information of phonemes and syllables.
02 to extract the local duration. Next, the process proceeds to step S502, where a phoneme / language environment file 503 having information about the phoneme environment obtained from phoneme information such as phonemes, the number of mora, the number of accent phrases, and the language environment obtained from language information such as part of speech, and step S501. Using the extracted information on the local duration, a local segment duration model 302 is created in consideration of a predetermined phonemic environment.

As a specific processing procedure, a method similar to that of the global duration model 301 of the global segment described above may be used. That is, a model in which the local duration is normalized using the average duration of the local segment obtained from the K learning samples is created, and the local duration model 302 is created based on this model. May be.

Finally, for the global segment obtained from the sum of the global duration for the global segment obtained in step S302 and the local duration for a plurality of local segments obtained in step S303. The difference from the global duration (for example, (600-300 =) 300 milliseconds in the above example) is used to calculate the global segment using statistics (mean, variance) relating to the duration of the phoneme. To be equal to the global duration,
In step S304, expansion / contraction processing is performed. This specific method can be realized by using a means such as an expansion / contraction method using a statistic related to the duration of a phoneme as disclosed in Japanese Patent Application Laid-Open No. H11-259095.

For example, as an example of determining a phoneme time length for a certain phoneme, an average value, a standard deviation, and a minimum value of the phoneme time length are obtained for each phoneme type (αi), and these are stored in a memory. Using these values, the initial value dαi of the phoneme duration di for the phoneme αi is determined. Then, the phoneme duration di is determined based on this.

Di = dαi + ρ (σαi) ² ρ = (T−Σdαi) / Σ (σαi) ² where T indicates the generation time (T = Σdi), and σαi indicates the standard deviation of the phoneme time length. Σ indicates the sum of i = 1 to N (the number of samples).

[Second Embodiment] In the first embodiment,
A model for estimating equation (1) obtained by dividing the duration dk of the global segment by the average duration d of the global segment to d is learned, and the local duration is obtained using the global duration obtained from this model. Although the duration time is reset, Embodiment 2
Then, a global time length model is constructed based on the difference between the duration of the global segment and the average duration. The hardware configuration and procedure according to the second embodiment are the same as those in the first embodiment.
Since these embodiments are the same as the first embodiment (FIGS. 1 to 5), the description thereof is omitted.

In the second embodiment, the equation (1) in the first embodiment is changed to sk = dk− ~ d... (5), and the average duration is calculated from the duration of the global segment for each learning sample. By subtracting the time length ｄd, the sk obtained by normalizing the duration time dk is obtained. Using the sk obtained in this manner, a prediction model of sk can be created in the same manner as in the first embodiment, using a linear multiple regression analysis method in the same manner as in equation (3). Using the predicted value of the duration of the global segment ^ sk obtained from this model, the duration of the global segment ^ dk for the k-th sample is given by d ^ k = ^ from equation (5). sk + ~ d can be obtained as equation (6). Equation (6) is a global duration model in the second embodiment. The local duration model can be modeled using a similar method.

The configuration in each of the above embodiments shows an embodiment of the present invention, and various modifications are possible. A modified example is as follows.

In each of the above embodiments, the average duration of the mora is used as the average duration of the global segment to d. However, when the average is obtained, the mora is used as a unit. Other phoneme units, such as phonemes, can be used. The present invention is also applicable to languages other than Japanese.

In the above-described embodiments, the factors and categories of the linear multiple regression model of the global segment are merely examples, and other factors and categories may be used.

Another object of the present invention is to provide a storage medium storing a program code of software for realizing the functions of the above-described embodiments to a system or apparatus, and to provide a computer (or CPU or MPU) of the system or apparatus. Is also achieved by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. As a storage medium for supplying such a program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-RO
M, CD-R, DVD, magnetic tape, nonvolatile memory card, ROM and the like can be used.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also the OS (Operating System) running on the computer based on the instruction of the program code. System) performs part or all of actual processing, and the processing realizes the functions of the above-described embodiments.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instructions of the program code, The case where the CPU of the function expansion board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments is also included.

As described above, according to the present embodiment, the duration can be modeled with higher accuracy by using the means for setting the duration of the global and local segments with high accuracy. As a result, it is possible to improve the naturalness of synthesized speech in the speech synthesis device.

[0042]

As described above, according to the present invention, it is possible to accurately set the duration of a phoneme sequence, and to provide a natural phoneme time according to the phoneme / language environment.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a hardware configuration of a speech synthesizer according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a speech synthesis processing procedure in the speech synthesis device according to the embodiment of the present invention.

FIG. 3 is a flowchart showing a procedure for setting a duration of a phoneme sequence using a duration model in a prosody generation process in step S203 of FIG. 2;

FIG. 4 is a flowchart illustrating a method for creating a global duration model for a global segment according to the present embodiment.

FIG. 5 is a flowchart showing a method for creating a local duration model for a local segment according to the present embodiment.

Claims (11)

[Claims] A step of obtaining a duration of a phoneme sequence in a predetermined unit based on a duration model of a global segment; and a step of obtaining each of the phoneme sequences based on a duration model of a local segment. Setting the duration of each phoneme based on the duration of the phoneme sequence and the duration of each phoneme, and setting the duration of each phoneme. A voice synthesizing step of synthesizing voice based on the duration of each phoneme. 2. The method according to claim 1, wherein the local segment comprises at least one of a phoneme, a syllable, and a mora, and the global segment comprises at least one of an accent phrase, a word, a phrase, and a sentence. The voice information processing method described in the above. 3. The global segment duration model is a model modeled on the basis of a ratio of the duration of the global segment to the average duration of the global segment. The voice information processing method according to claim 1. 4. The duration model of the global segment is a model modeled based on a difference between the duration of the global segment and the average duration of the global segment. The voice information processing method according to claim 1. 5. The method according to claim 1, wherein the duration model of the global segment is a model modeled by a linear multiple regression model.
The voice information processing method according to the paragraph. 6. A computer-readable storage medium storing a program for executing the voice information processing method according to claim 1. Description: 7. A means for determining the duration of a phoneme sequence of a predetermined unit based on a duration model of a global segment; and each of the units forming the phoneme sequence based on a duration model of a local segment. Means for determining the duration of a phoneme; setting means for setting the duration of each phoneme based on the duration of the phoneme sequence and the duration of each phoneme; and setting by the setting means. Speech synthesis means for synthesizing speech based on the duration of each phoneme. 8. The method of claim 7, wherein the local segment comprises a phoneme or at least one of a syllable and a mora, and the global segment comprises at least one of an accent phrase, a word, a phrase, and a sentence. An audio information processing apparatus according to claim 1. 9. The global segment duration model is a model modeled based on a ratio of a duration of the global segment to an average duration of the global segment. The voice information processing apparatus according to claim 7, wherein 10. The duration model of the global segment is a model modeled based on a difference between the duration of the global segment and the average duration of the global segment. The voice information processing apparatus according to claim 7, wherein 11. The speech information processing apparatus according to claim 7, wherein the duration model of the global segment is a model modeled by a linear multiple regression model.