CN116189680B - Voice wake-up method of exhibition intelligent equipment - Google Patents

Abstract

The invention provides a voice wake-up method of exhibition intelligent equipment, which relates to the technical field of intelligent voice interaction, and comprises the steps that the intelligent equipment receives voice with the highest decibel in a preset region range and preset time and face dynamic images of all users; judging whether the first user is still in a preset region range currently according to the face dynamic image of the first user; and locking the face dynamic image of the first user, performing first voice interaction with the first user, extracting first voiceprint features of the first user, and filtering voiceprint features which are not matched with the first voiceprint features based on the first voiceprint features. The voice wake-up method of the exhibition intelligent equipment solves the technical problem that the interaction pertinence is insufficient when the existing intelligent equipment communicates with people in a noisy environment.

Description

A voice wake-up method for displaying smart devices

Technical field

The present invention relates to the technical field of intelligent voice interaction, and in particular to a voice wake-up method for performing intelligent equipment.

Background technique

Intelligent audio guides for exhibitions are devices that provide voice explanations of indoor exhibition objects to facilitate visitors to have an in-depth understanding of the exhibition objects. They are widely used in the field of audio guides.

The currently commonly used intelligent audio guide is that after the user manually turns it on, it searches for and broadcasts fixed content in the question bank according to the user's questions. The content played does not change due to the change of the user, nor can it be interacted with a specific visitor. Targeted interactions. In the field of smartphones, intelligent voice assistants can wake up according to the user's wake-up word and start a human-machine conversation with the user. However, this method is generally used for conversations between a single person and a machine. In complex scenes with loud noise and many people, such as exhibition scenes, there is a problem of low content recognition accuracy and difficulty in identifying the speaking subject, making it difficult for smart devices to recognize Interaction objects cannot understand instructions well during interaction, resulting in untargeted interaction. Therefore, existing smart voice assistants are difficult to use in crowded and noisy environments such as exhibitions and exhibitions.

Therefore, an improved voice wake-up method for performing smart devices is urgently needed to improve the above technical problems.

Contents of the invention

In view of this, the purpose of the present invention is to provide a voice awakening method for a presentation smart device that can well lock the interactor during human-computer interaction in a crowded and noisy environment, and thereby enable targeted responses.

The invention provides a voice wake-up method for performing intelligent equipment, which includes: the intelligent equipment receives the voice with the highest decibel in a preset geographical range and within a preset time and the dynamic images of faces of all users; the voice with the highest decibel is the first voice , perform semantic analysis on the first speech to obtain the first semantics; perform image analysis on the facial dynamic images of all users to obtain a facial dynamic image set, and obtain lip dynamic images corresponding to the facial dynamic image collection through image extraction Set, obtain the second semantic set corresponding to the lip dynamic image set through lip language analysis; if the first semantics is in the second semantics set, extract the third semantics corresponding to the first semantics from the second semantics set A dynamic image of a user's lips and a dynamic image of the first user's face; if the first semantics is not in the second semantic set, re-receive the voice with the highest decibel in the preset geographical range and within the preset time; according to the first The dynamic image of the user's face determines whether the first user is currently in the preset geographical range. If not, re-receive the voice with the highest decibel in the preset geographical range and within the preset time; if so, lock the first user's face. dynamic image, and perform the first voice interaction with the first user, extract the first voiceprint feature of the first user, and filter the voiceprint features that do not match the first voiceprint feature based on the first voiceprint feature .

Preferably, the voice awakening method of a performance smart device provided by the present invention also includes: establishing a user database. The user database includes a plurality of user characteristic information, and the user characteristic information includes age, gender, accent, and voice interaction records.

Preferably, after the first voice interaction, the first voiceprint feature is compared with the user feature information of the user database; the user feature information is further divided into manager data information and visited user data information. The manager data information corresponds to the manager's voice interaction record, and the accessed user data information corresponds to the accessed user interaction record; if it is determined that the first voiceprint feature is the manager's data information, the manager's voice interaction record is retrieved. In the next interaction, if it is determined that the first voiceprint feature is the visited user data information, the visited user interaction record is retrieved for the next interaction.

Preferably, the age and gender are extracted from the face dynamic image collection; the speech recognition method of the accent includes: constructing a syllable mapping table from Mandarin pronunciation to dialect pronunciation according to the characteristics of a specific dialect; according to the syllable mapping table, Extend the existing standard Mandarin speech recognizer and form a first search tree; replace the search tree in the standard Mandarin speech recognizer with the first search tree and form a second search tree.

Preferably, the voice with the highest decibel includes a wake-up instruction.

Preferably, the lip language analysis method is: obtaining lip motion feature data from the lip dynamic image; determining the forward standard deviation and/or inverse standard deviation of the lip motion feature data; based on the forward standard deviation and /or reverse standard deviation to determine the word segmentation result of the second semantic set corresponding to the lip dynamic image set.

Preferably, the lip movement feature data includes: the upper lip characteristic angle and upper lip area formed by the left lip angle, right lip angle, and upper lip peak, and the lower lip characteristic angle formed by the left lip angle, right lip angle, and lower lip low point. and lower lip area.

Preferably, determining the forward standard deviation of the lip movement feature data includes: selecting a first visual frame to determine the lip dynamic image, the first visual frame and the lip movements of the previous frame image of the first visual frame. Characteristics are determined; determining the inverse standard deviation of the lip movement feature data includes: selecting and determining the first visual frame of the lip dynamic image, the first visual frame and the lip movements of the rear view frame image of the first visual frame Characteristics are determined.

Preferably, the method of filtering voiceprint features that do not match the first voiceprint feature is: the smart device includes a microphone array, a ToF detection module, and a DOA calculation module; the microphone array processes multi-channel voice signals, and performs degradation on the voice signals. Noise and enhancement; the ToF detection module detects people in the preset geographical range and generates person location information; the DOA calculation module calculates and generates the current DOA interval data; calculates the data input based on the microphone array and the data generated based on the DOA calculation module data, filtering out voiceprint features that do not match the first voiceprint feature.

Preferably, the method for semantically analyzing the first speech to obtain the first semantics includes: defining all terminal symbols, non-terminal symbols and rule classifications in the grammar according to the domain task, the terminal symbols are keywords classified according to semantics, and the key Words can contain Arabic numerals and English letters. Each keyword has a corresponding pinyin. Each rule is assigned a priority level. The priority set of rules is obtained through lexical analysis or non-lexical analysis. The rules are related to semantics. Directly related, each rule corresponds to a semantic analysis function, which reads the context-independent enhanced grammar based on semantic classes from the grammar configuration file; performs word segmentation on the sentences input by the user; performs syntactic analysis on the word segmentation results; and obtains the optimal syntax. The analysis results are subjected to semantic analysis to obtain the user's final search keyword information.

The technical solution of the present invention brings the following beneficial effects: In the voice wake-up method of a display smart device provided by the present invention, taking use at an exhibition venue as an example, it includes the following steps: the smart device searches for a preset geographical range and a preset Timely venue information, receive dynamic facial images of all users with the voice of the person with the highest decibel level within the preset time and preset geographical range. The voice with the highest decibel is set as the first voice, and semantic analysis is performed on the first voice to obtain the first semantics corresponding to the first voice. At the same time, image analysis is performed on all users' dynamic facial images to obtain a facial dynamic image set, and a lip dynamic image set corresponding to the facial dynamic image set is obtained through image extraction means, and a lip dynamic image set is obtained through lip language analysis. The second semantic set corresponding to the image set.

After obtaining the first semantics and the second semantic set by processing the voice of the person with the highest decibel voice and the face dynamic images of all users in the preset geographical range and within the preset time, it is determined whether the above-mentioned first semantics is in the first semantic set. Within the two semantic sets, if the first semantics is in the second semantics set, the first semantics of the person with the highest decibel voice, the face dynamic image, and the lip dynamic image are obtained, and the characteristic information of the person you are looking for is locked and obtained. . If not, it means that the recognition noise is large in a noisy environment, and the smart device will re-receive the voice with the highest decibel in the preset geographical range and within the preset time.

Determine whether the first user is currently still in the preset geographical range based on the dynamic image of the first user's face. If not, the first user may have left the preset geographical range, and the smart device will receive the preset geographical range and decibels within the preset time again. The highest voice. If so, lock the dynamic image of the first user's face, conduct the first voice interaction with the first user, extract the first voiceprint feature of the first user, and filter the first voiceprint feature based on the first voiceprint feature Mismatched voiceprint features. If the first user is still present at this time, the first voiceprint feature of the first user is extracted for more targeted interaction, and voiceprint features that do not match the first voiceprint feature are filtered out.

Based on this, the present invention provides a voice awakening method for performing smart devices. By accurately searching and identifying the first user, the first semantics, face dynamic images, and lip dynamic images of the first user are successively obtained. As well as the first voiceprint feature, even in a noisy environment, it can find users in a targeted manner and conduct targeted communication based on user characteristics, improving the user experience.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description, claims and appended drawings.

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, preferred embodiments are given below and described in detail with reference to the accompanying drawings.

Description of the drawings

In order to more clearly explain the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings that need to be used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description The drawings illustrate some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a structural block diagram of a method for displaying voice wake-up of an intelligent device according to an embodiment of the present invention.

Figure 2 is a structural block diagram of another voice wake-up method for displaying smart devices provided by an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present invention.

In order to facilitate understanding of this embodiment, a control device disclosed in the embodiment of the present invention is first introduced in detail.

The present invention provides a voice wake-up method for performing smart devices, as shown in Figure 1. The method includes the smart device receiving the voice with the highest decibel in a preset geographical range and within a preset time and the face dynamic images of all users; the highest decibel is received by the smart device. The voice of the user is the first voice, and semantic analysis is performed on the first voice to obtain the first semantics; image analysis is performed on the dynamic face images of all users to obtain a set of dynamic face images, and the lips corresponding to the set of dynamic face images are obtained through image extraction. A set of lip dynamic images is obtained, and a second semantic set corresponding to the set of lip dynamic images is obtained through lip language analysis.

This embodiment provides a voice wake-up method for a performance smart device, taking use at an exhibition venue as an example, including the following steps: Step S110: The smart device searches for venue information within a preset geographical range and within a preset time, and receives the information at the preset time. Dynamic facial images of all users within the preset geographical range and the voice of the person with the highest decibel sound. The voice with the highest decibel is set as the first voice, and semantic analysis is performed on the first voice to obtain the first semantics corresponding to the first voice. At the same time, image analysis is performed on all users' dynamic facial images to obtain a facial dynamic image set, and a lip dynamic image set corresponding to the facial dynamic image set is obtained through image extraction means, and a lip dynamic image set is obtained through lip language analysis. The second semantic set corresponding to the image set.

After obtaining the first semantic set and the second semantic set by processing the voice of the person with the highest decibel voice and the face dynamic images of all users in the preset geographical range and within the preset time, step S120 is performed.

Step S120: Determine whether the above-mentioned first semantics is in the second semantic set. If the first semantics is in the second semantic set, the first semantics, facial dynamic images, and lip dynamics of the person with the highest decibel sound are obtained. image, lock and obtain the characteristic information of the person you are looking for. If not, it means that the recognition noise is large in a noisy environment, and the smart device will re-receive the voice with the highest decibel in the preset geographical range and within the preset time, and start step S110 again.

Step S130: Determine whether the first user is currently still in the preset geographical range based on the first user's facial dynamic image. If not, the first user may have left the preset geographical range, and the smart device receives the preset geographical range and preset settings again. the highest decibel voice within the time period and restart step S110. If yes, continue to step S140.

Step S140: Lock the dynamic face image of the first user, conduct the first voice interaction with the first user, extract the first voiceprint feature of the first user, and filter the first voiceprint feature based on the first voiceprint feature Mismatched voiceprint features. If the first user is still present at this time, the first voiceprint feature of the first user is extracted for more targeted interaction, and voiceprint features that do not match the first voiceprint feature are filtered out. Based on this, this embodiment provides a voice awakening method for a performance smart device. By accurately searching and identifying the first user, the first semantics, facial dynamic images, and lip dynamic images of the first user are successively obtained. As well as the first voiceprint feature, even in a noisy environment, it can find users in a targeted manner and carry out targeted communication based on user characteristics, improving the user experience.

Further, the voice awakening method of a performance smart device provided by this embodiment also includes step S150: establishing a user database. The user database includes multiple user characteristic information, and the user characteristic information includes age, gender, accent, and voice interaction records.

Further, step S150 further includes step S151: further dividing the user characteristic information into manager data information and visited user data information. The manager data information corresponds to the manager voice interaction record, and the visited user data information corresponds to Visited user interaction records.

Step S152: If it is determined that the first voiceprint feature is the administrator's data information, retrieve the administrator's voice interaction record for the next interaction. If it is determined that the first voiceprint feature is the visited user data information, retrieve the visited user interaction record for the next interaction. Next interaction.

After it is determined in step S140 that the first user still needs further interaction within the preset range, a user database is established in step S150. And through step S151, the user characteristics of the user database are further divided into manager data information and visited user data information. If it is determined that the first voiceprint feature matches the manager data information, the manager's characteristics and question and answer records are called, and targeted sexual answer. If it is determined that the first voiceprint feature matches the visited user data information, the visited user data information is called for interaction, making the entire interaction process more targeted. In the eyes of the first user, the interaction process of the displayed smart device is more intelligent.

This embodiment provides a voice wake-up method for performing smart devices, which also includes: establishing a user database. The user database includes multiple user characteristic information. The user characteristic information includes age, gender, accent, and voice interaction records, where age and gender Extracted from a collection of face dynamic images; the accent speech recognition method includes: building a syllable mapping table from Mandarin pronunciation to dialect pronunciation according to the characteristics of a specific dialect; extending the existing standard Mandarin speech recognizer based on the syllable mapping table, and Form a first search tree; replace the search tree in the standard Mandarin speech recognizer with the first search tree and form a second search tree. According to the characteristics of a specific dialect, the method of constructing a syllable mapping table from Mandarin pronunciation to dialect pronunciation includes: summarizing the syllable mapping rules of relevant dialects based on language knowledge; for any syllable mapping that is unrelated to a word, if the mapping occurs on the initial consonant, register The initial consonant mapping pair {I*(x)}→{I*(y)} means that the initial consonant of a syllable containing the initial consonant x will be mapped to y. This makes the voice wake-up method for displaying smart devices provided in this embodiment wider applicable.

Further, the voice with the highest decibel in step S110 includes a wake-up instruction, and the wake-up instruction includes wake-up keywords, wake-up sentence patterns, interrogative modal words, etc. This embodiment provides a voice wake-up method for performing smart devices after finding the first user. Later, wake-up instructions can also be used during the interaction to ensure timely communication with the first user.

Further, a second semantic set corresponding to the lip dynamic image set is obtained through lip language analysis. The lip language analysis method is: obtaining lip movement feature data from the lip dynamic image; determining the forward standard of the lip movement feature data Difference and/or inverse standard deviation; based on the above forward standard deviation and/or inverse standard deviation, determine the word segmentation result of the second semantic set corresponding to the above lip dynamic image set.

The lip movement feature data includes: the characteristic angle of the upper lip and the area of the upper lip composed of the left lip angle, the right lip angle and the upper lip peak, and the characteristic angle and area of the lower lip composed of the left lip angle, right lip angle and the low point of the lower lip. . The characteristic angle of the upper lip and the area of the upper lip represent the characteristics of the user's upper lip when speaking. The characteristic angle of the lower lip and the area of the lower lip represent the characteristics of the user's lower lip when speaking. When users speak different contents, they have a set of upper lip characteristics and lower lip characteristics. .

Further, determining the forward standard deviation of the lip motion feature data includes: selecting the first video frame to determine the lip dynamic image, and determining the lip motion features of the first video frame and the previous frame image of the first video frame. ; Determining the inverse standard deviation of the lip motion feature data includes: selecting the first video frame to determine the lip dynamic image, and determining the lip motion features of the first video frame and the subsequent video frame image of the first video frame. According to the dynamically determined forward standard deviation of the lip motion feature data and the inverse standard deviation of the lip motion feature data, the word segmentation result of the second semantic set corresponding to the lip dynamic image set can be determined, thereby identifying the user's interactive content.

Further, a method for filtering voiceprint features that do not match the first voiceprint feature based on the first voiceprint feature is: the smart device includes a microphone array, a ToF (time of flight) detection module, and a DOA (direction of arrival) calculation module; The microphone array processes multi-channel voice signals, denoising and enhancing the voice signals; the ToF detection module detects people in the preset geographical range, and generates person location information. The DOA calculation module generates the current DOA interval data; the calculation is based on the above microphone array input. Data and based on the data generated by the DOA calculation module, the voiceprint features that do not match the first voiceprint feature are filtered.

The present invention provides a method for voice wake-up of intelligent equipment for exhibitions that detects the status and location information of people in the exhibition through the ToF detection module, and then inputs it to the DOA calculation module to calculate and generate the current DOA interval data, thereby making the DOA calculation during voice wake-up more accurate. , reducing DOA calculation errors, so that subsequent speech noise reduction targets are accurate, ultimately improving the recognition accuracy and improving user experience.

Further, the method of semantically analyzing the first speech to obtain the first semantics includes: first, reading the context-independent enhanced grammar based on semantic classes from the grammar configuration file. Specifically, defining all terminal symbols in the grammar according to the domain task, Classification of non-terminal symbols and rules. Terminal symbols are keywords classified according to semantics. Keywords can include Arabic numerals and English letters. Each keyword has a corresponding pinyin. Each rule is assigned a priority level. Priority The set of rules is obtained through lexical analysis or non-lexical analysis. The rules are directly related to semantics. Each rule corresponds to a semantic analysis function. Among them, domain task definition grammar is an existing language in semantic analysis.

Then, perform word segmentation on the sentence input by the user; perform syntactic analysis on the word segmentation results; perform semantic analysis on the optimal syntactic analysis results to obtain the user's final search keyword information.

The invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in one process or multiple processes of the flowchart and/or one block or multiple blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the invention. In this way, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and equivalent technologies, the present invention is also intended to include these modifications and variations.

In addition, in the description of the embodiments of the present invention, unless otherwise clearly stated and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense. For example, it can be a fixed connection or a detachable connection. , or integrally connected; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be an internal connection between two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood in specific situations.

In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description. It does not indicate or imply that the device or element referred to must have a specific orientation or a specific orientation. construction and operation, and therefore should not be construed as limitations of the invention. Furthermore, the terms “first”, “second” and “third” are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

Finally, it should be noted that the above embodiments are only specific implementations of the present invention and are used to illustrate the technical solutions of the present invention rather than to limit them. The protection scope of the present invention is not limited thereto. Although refer to the foregoing embodiments The present invention has been described in detail. Those skilled in the art should understand that any person familiar with the technical field can still modify the technical solutions recorded in the foregoing embodiments or can easily think of them within the technical scope disclosed by the present invention. changes, or equivalent substitutions of some of the technical features; these modifications, changes or substitutions do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be covered by the protection scope of the present invention. Inside. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (9)

1. A voice wake-up method for performing smart devices, which is characterized by including: The smart device receives the voice with the highest decibel in a preset geographical range and within a preset time and the dynamic images of faces of all users; the voice with the highest decibel is the first voice, and semantic analysis is performed on the first voice to obtain the first semantics; Perform image analysis on the facial dynamic images of all users to obtain a facial dynamic image set, obtain a lip dynamic image set corresponding to the facial dynamic image set through image extraction, and obtain a lip dynamic image set corresponding to the lip language analysis. The second semantic set corresponding to the dynamic image set; If the first semantics is in the second semantics set, extract the first user's lip dynamic image and the first user's face dynamic image corresponding to the first semantics from the second semantics set; If the first semantics is not in the second semantics set, re-receive the voice with the highest decibel in the preset geographical range and within the preset time; Determine whether the first user is currently still in the preset geographical range based on the dynamic image of the first user's face. If not, re-receive the voice with the highest decibel in the preset geographical range and within the preset time; if so, lock the location. and perform the first voice interaction with the first user, extract the first voiceprint feature of the first user, and filter the interaction with the first voiceprint feature based on the first voiceprint feature. The first voiceprint feature does not match the voiceprint feature; The method of filtering voiceprint features that do not match the first voiceprint feature is: the smart device includes a microphone array, a ToF detection module, and a DOA calculation module; the microphone array processes multiple channels of voice signals, and degrades the voice signals. Noise and enhancement; the ToF detection module detects people in the preset geographical range and generates person location information; the DOA calculation module calculates and generates the current DOA interval data; calculates based on the data input by the microphone array and based on the The data generated by the DOA calculation module is used to filter out voiceprint features that do not match the first voiceprint feature. 2. The voice wake-up method for displaying smart devices according to claim 1, further comprising: Establish a user database, which includes multiple user characteristic information, including age, gender, accent, and voice interaction records. 3. The voice wake-up method of the performance smart device according to claim 2, characterized in that, after the first voice interaction, the first voiceprint feature is compared with the user feature information of the user database. Comparison; The user characteristic information is further divided into manager data information and visited user data information. The manager data information corresponds to the manager voice interaction record, and the visited user data information corresponds to the visited user interaction record; If it is determined that the first voiceprint feature is the data information of the manager, the voice interaction record of the manager is retrieved for the next interaction; if it is determined that the first voiceprint feature is the data information of the visited user, the voice interaction record of the manager is retrieved for the next interaction. Get the visited user interaction record to perform the next interaction. 4. The voice awakening method of the performance smart device according to claim 2, characterized in that the age and the gender are extracted from the face dynamic image collection; the voice recognition method of the accent includes: according to a specific According to the characteristics of dialects, a syllable mapping table from Mandarin pronunciation to dialect pronunciation is constructed; according to the syllable mapping table, the existing standard Mandarin speech recognizer is expanded and a first search tree is formed; the first search tree is used to replace the search tree in a standard Mandarin speech recognizer and form a second search tree. 5. The voice wake-up method for displaying smart devices according to claim 1, wherein the voice with the highest decibel includes a wake-up instruction. 6. The voice awakening method of the performance smart device according to claim 1, characterized in that the lip language analysis method is: Obtain lip motion feature data from the lip dynamic image; Determine the forward standard deviation and/or inverse standard deviation of the lip movement characteristic data; Based on the forward standard deviation and/or the inverse standard deviation, a word segmentation result of the second semantic set corresponding to the lip dynamic image set is determined. 7. The voice awakening method of the performance smart device according to claim 6, wherein the lip movement characteristic data includes: the characteristic angle of the upper lip formed by the left lip angle, the right lip angle, and the upper lip peak, and the upper lip area, The characteristic angle of the lower lip and the area of the lower lip formed by the left lip corner, the right lip corner and the low point of the lower lip. 8. The voice awakening method of a performance smart device according to claim 6, wherein determining the forward standard deviation of the lip motion feature data includes: selecting and determining the first frame of the lip dynamic image, the The lip motion features of the first video frame and the previous frame image of the first video frame are determined; determining the inverse standard deviation of the lip motion feature data includes: selecting and determining the first lip motion feature of the dynamic image. The lip motion characteristics of the first video frame and the subsequent video frame image of the first video frame are determined. 9. The voice awakening method of a performance smart device according to claim 1, wherein the method of performing semantic analysis on the first voice to obtain the first semantics includes: All terminal symbols, non-terminal symbols and rules in the grammar are classified according to the domain task definition. The terminal symbols are keywords classified according to semantics. The keywords include Arabic numerals and English letters. Each keyword has a corresponding pinyin. , each of the rules is assigned a priority level, and the priority rule set is obtained through lexical analysis or non-lexical analysis. The rules are directly related to semantics. Each of the rules corresponds to a semantic analysis function. , read the context-independent enhanced grammar based on semantic classes from the grammar configuration file; perform word segmentation on the sentences entered by the user; perform syntactic analysis on the word segmentation results; perform semantic analysis on the optimal syntactic analysis results to obtain the user's final search keywords information.