Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating a task scheduling method for video rendering according to an embodiment of the present application, where in the embodiment, the method for task scheduling for video rendering includes:
step 101, distinguishing the type of a video rendering task, processing the video rendering task according to the type of the video rendering task, and synchronizing the video rendering task to a distributed task scheduling platform.
In this embodiment, a user creates, starts, pauses, and stops tasks at a user side according to a service requirement of the user, distinguishes the type of a video rendering task according to a service scene, processes the video rendering task according to the type of the video rendering task, and directly outputs a fund video, a weekly report of the fund, a quarterly report description, and retries after a video rendering failure. And processing the video rendering task is realized through a task processing restful interface of the distributed task scheduling platform, and the task created by the interface service in the project is synchronized to the distributed task scheduling platform through an mq (Message Queue) Message in the task distributor, so that the video rendering task is synchronized to the distributed task scheduling platform and the attribute of the video rendering task is kept consistent, and the creation, the stop and the deletion of the video rendering task are realized. The business scenario is that enterprises and merchants need to provide products or services which may be needed and associated by consumers in a certain specific link of a user at a proper time. The restful interface is a design style and development mode of a network application program, and can be defined by an XML format or a JSON format based on HTTP. RESTFUL is suitable for a scene that a mobile internet manufacturer serves as a service interface, the function that a third party OTT calls mobile network resources is achieved, and the action type is to add, change and delete the called resources. The mq message is a "first-in-first-out" data structure in the underlying data structure. The method is generally used for solving the problems of application decoupling, asynchronous messages, flow peak clipping and the like, and realizes a high-performance, high-availability, scalable and final consistency framework.
And 102, acquiring a video engine of the video rendering task, and acquiring an available engine address according to the video engine.
In this embodiment, the video engines of the video rendering tasks are distinguished according to the service areas, the contents of the video engines of different video rendering tasks are different, and an available engine address is obtained according to the video engines, so that the video rendering tasks can be rendered stably and efficiently by the video engine at the rear end under the condition that hardware resources are limited.
And 103, processing the video rendering task according to the engine address to obtain a processing result.
In this embodiment, the video rendering task is processed according to the acquired available engine address, so that a processing result of the video engine can be obtained.
And 104, processing the processing result.
In this embodiment, the processing result of the video rendering task is processed again.
The embodiment of the application provides a task scheduling method for video rendering, which comprises the following steps: distinguishing the type of a video rendering task, processing the video rendering task according to the type of the video rendering task and synchronizing the video rendering task to a distributed task scheduling platform; acquiring a video engine of the video rendering task, and acquiring an available engine address according to the video engine; processing the video rendering task according to the engine address to obtain a processing result; and processing the processing result. By the method provided by the invention, different video rendering tasks can be realized by the same actuator, different engine addresses can be obtained according to the execution characteristics of different tasks, the video rendering tasks can be realized, the stable and efficient rendering and video output of a rear-end video engine can be ensured, the throughput and the execution efficiency of the tasks can be horizontally expanded, the task processing capacity can be elastically expanded along with resource allocation, the service decoupling can be realized, the high performance and the service modularization of the service are realized, and the expansibility of the service is improved.
Further, referring to fig. 2, fig. 2 is a sub-flow diagram of a task scheduling method for video rendering in the embodiment of the present application, and in this embodiment, acquiring a video engine of the video rendering task, and acquiring an available engine address according to the video engine specifically includes:
step 201, obtaining a video engine of the video rendering task according to the type of the video rendering task, wherein the type of the video rendering task comprises an immediate execution task, a periodic execution task and a task to be executed;
and step 202, distributing the engine address according to the state and the task execution number of the video engine, wherein the video engine at least comprises an online engine and an office engine.
In this embodiment, the types of video rendering tasks distinguished according to a service scene are respectively an immediate execution task, a periodic execution task, and a task to be executed, and are respectively a video of a direct output fund, a weekly report of the fund, a quarterly report description, and retry processing after a video rendering failure, a video engine of the video rendering task is acquired according to the type of the video rendering task, and an engine address is allocated according to the state of the video engine and the current task execution number of the video engine, where the video engine includes an online engine and an offine engine, the online engine is responsible for a video rendering task request initiated directly, the offine engine is responsible for the periodic task processing and the task processing to be processed, and an available engine address is allocated to the online engine and the offine engine according to the amount of the task.
Further, the allocation mode at least comprises one of a polling mode, a weighting mode or a custom allocation mode.
In the present embodiment, the polling mode: and confirming whether the tasks are distributed in an online engine or an office engine according to different types of tasks, acquiring the addresses of the assignable video engines according to the states of the engine addresses and the current execution numbers of the engine addresses, and ensuring that all requests can be distributed to the executable engine addresses for processing through a polling algorithm. The weighting mode is as follows: and distinguishing multiple engine instances according to the number of CPUs (central processing units) providing the video engine server, and ensuring that the request is preferentially distributed to a server with high configuration for processing. A user-defined distribution mode: in a dispatching center in a distributed task dispatching center, a self-defined distribution mode can be adopted, and the dispatching center comprises routing strategies, polling, randomness, consistent hash, fault transfer or task fragmentation and the like, and blocking processing strategies, such as: single machine serialization, discard subsequent scheduling, override pre-scheduling, etc.
In this embodiment, the polling mode and the weighting mode may switch processing between acquiring an available engine address and executing the engine according to the type of the video rendering task.
Further, referring to fig. 3, fig. 3 is another sub-flow diagram of the task scheduling method for video rendering in the embodiment of the present application, where processing the video rendering task according to the engine address to obtain a processing result specifically includes:
301, processing the video rendering task in batches by using multiple threads;
step 302, asynchronously acquiring the processing result through CompletableFuture and custom thread, and intervening in log recording and information notification.
In the embodiment, the video rendering task is processed in batches through multiple threads, the video engine is called in batches based on high performance service, the processing result is also asynchronously obtained through CompletableFuture and custom ThreadLocal, the obtained processing result is recorded in a log, and the user is informed of the obtained processing result through information, wherein the CompletableFuture is a new characteristic of Java 8.
Further, referring to fig. 4, fig. 4 is a schematic view of another sub-flow of a task scheduling method for video rendering in the embodiment of the present application, where processing the processing result specifically includes:
step 401, if the processing result is rendering failure, solidifying the basic attribute of the video rendering task to mysql, and continuing to acquire the available engine address to process the video rendering task until rendering is successful;
and 402, if the rendering is successful, storing the video data of the video rendering task to redis, and recording the task record and the execution details of the video rendering task.
In this embodiment, if a video rendering task that fails to be called by an engine address and an engine address is not obtained when an available engine address is obtained to process the video rendering task, a task failure queue is newly added, a failure record and a basic attribute of a video of the video rendering task are all fixed to mysql (relational database management system), and a custom task discarding policy and a failure retry are added, where the basic attribute of the video includes: the mysql is a relational database management system, and the relational database stores data in different tables, rather than putting all data in a large warehouse, thus increasing speed and flexibility. And after the basic attribute of the video is solidified to mysql, the available engine is obtained through the data compensation task to continue processing the video rendering task until the video rendering task is successfully rendered.
In this embodiment, after the video rendering task is successfully rendered, the video data of the video rendering task is saved to redis, and records the task and the record of the execution details, such as an engine address, an execution state, a task execution number, an update time, and the like, where redis is a high-performance open-source key-value database, has a high read-write performance, and can store mappings (mapping) between keys (keys) and values (values) of 5 different types, and persist the key value pair data stored in the memory to a hard disk, and a user can configure different persistence modes according to an actual usage scenario.
Further, referring to fig. 5, fig. 5 is a schematic sub-flow diagram of a task scheduling method for video rendering in the embodiment of the present application, where if the processing result is rendering failure, curing the basic attribute of the video rendering task to mysql, and continuously acquiring the available engine address to process the video rendering task, further includes:
step 501, if the available engine address is not obtained within the preset time, performing exception message notification processing;
step 502, if the rendering failure times exceed preset failure times, discarding the video rendering task.
In this embodiment, if the processing result of the video rendering task is failure and the available engine is continuously acquired through the data compensation task to process the video rendering task, if the available engine address is not acquired within the preset time, the processing of the abnormal message notification is performed on the user, for example, a mail notification or a nail notification is performed through an enterprise mailbox; or if the number of times exceeds the preset number and the available engine address is not acquired, the video rendering task is discarded.
Further, if the processing result is rendering failure, curing the basic attribute of the video rendering task to mysql, and continuously acquiring the available engine address to process the video rendering task, further comprising:
and recording the update task record and the failure times of the video rendering task with rendering failure.
In this embodiment, after the video rendering task is completed, the video that has failed to be executed or has been blocked is recalled to the task distributor, and the task record and the number of failures are updated.
In this embodiment, a plurality of dynamic expansion tasks are integrated and abstracted according to characteristics of a video rendering task and processing results of the video rendering task, the plurality of dynamic expansion tasks are the same as the characteristics of the video rendering task and the processing results of the video rendering task, at present, two fixed tasks and a universal template task are dynamically expanded according to the processing results of the video rendering task and the video rendering task, a task has a single responsibility, one task corresponds to one trigger and one actuator and is used for realizing data processing after video rendering, the fixed tasks include file data processing and task compensation-processing to-be-executed tasks, wherein the file data processing is to process videos which have been rendered in redis, and a video engine maintains video rendering progress and maintains engine execution information such as: engine address, execution state, task execution number and update time. Data processing contains the basic attributes of a video such as: and storing data such as video addresses, video covers, video sizes, xml files described by videos, file names and the like into the mysql. The task compensation-processing task to be processed is a task which is processed without obtaining an engine address and failing in engine calling, a task failure queue is added, a custom task discarding strategy and failure retry are added to a failure record solidification database, after an available video engine is not obtained in more than ten minutes, an enterprise mailbox is used for carrying out mail notification or nail notification, the allowable failure times are specified, and the task is discarded after the failure times are exceeded.
The universal template task is a video rendering method for creating a task carrier and abstracting a core, and different task parameters are appointed to realize that n triggers correspond to one executor, such as video rendering of weekly reports and quarterly reports of funds, so as to realize the rendering processing of custom funds. The video rendering period task is a general template task, and specific task parameters are specified when the template task is created, such as: the execution period of the custom cron (plan task) expression, the associated user information, the project attribute, the file attribute, the code of the fund or the stock and the like are used for video rendering. The template task realizes different tasks and the same executor, namely, different dispatching centers of the same executor (jobHandler) are embodied in the distributed task dispatching platform.
Further, an embodiment of the present application further provides a video rendering task scheduling system 600, and fig. 6 is a schematic diagram of program modules of the video rendering task scheduling system in the embodiment of the present application, in which in this embodiment, the video rendering task scheduling system 600 includes:
the distinguishing module 601: the system comprises a distributed task scheduling platform, a video rendering task processing module, a video processing module and a video processing module, wherein the distributed task scheduling platform is used for processing video rendering tasks and synchronizing the video rendering tasks to the distributed task scheduling platform according to the types of the video rendering tasks;
the obtaining module 602: the video engine is used for acquiring the video rendering task and acquiring an available engine address according to the video engine;
the rendering module 603: the video rendering task is processed according to the engine address to obtain a processing result;
the processing module 604: for processing the processing result.
The embodiment of the application provides a task scheduling system 600 for video rendering, which can realize: distinguishing the type of a video rendering task, processing the video rendering task according to the type of the video rendering task and synchronizing the video rendering task to a distributed task scheduling platform; acquiring a video engine of the video rendering task, and acquiring an available engine address according to the video engine; processing the video rendering task according to the engine address to obtain a processing result; and processing the processing result. By the method provided by the invention, different video rendering tasks can be realized by the same actuator, different engine addresses can be obtained according to the execution characteristics of different tasks, the video rendering tasks can be realized, the stable and efficient rendering and video output of a rear-end video engine can be ensured, the throughput and the execution efficiency of the tasks can be horizontally expanded, the task processing capacity can be elastically expanded along with resource allocation, the service decoupling can be realized, the high performance and the service modularization of the service are realized, and the expansibility of the service is improved.
Further, the present application also provides a task scheduling apparatus for video rendering, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the steps in the task scheduling method for video rendering as described above.
Further, the present application also provides a storage medium, on which a computer program is stored, and when being executed by a processor, the computer program implements the steps in the task scheduling method for video rendering as described above.
Each functional module in the embodiments of the present invention may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a computer readable storage medium.
Based on such understanding, the technical solution of the present invention, which is described in the specification or contributes to the prior art in essence, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present invention is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no acts or modules are necessarily required of the invention. In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
For those skilled in the art, according to the idea of the embodiments of the present application, there may be variations in the specific implementation and application scope, and in summary, the content of the present description should not be construed as a limitation to the present invention.