Message queue protocol for sequential execution of related task routines in many task computing
Gespeichert in:
| Titel: | Message queue protocol for sequential execution of related task routines in many task computing |
|---|---|
| Patent Number: | 11762,689 |
| Publikationsdatum: | September 19, 2023 |
| Appl. No: | 18/091672 |
| Application Filed: | December 30, 2022 |
| Abstract: | An apparatus including a processor to: output a first request message onto a group sub-queue shared by multiple task containers to request execution of a first task routine; within a task container, respond to the first request message, by outputting a first task in-progress message onto an individual sub-queue not shared with other task containers to accede to executing the first task routine, followed by a task completion message; and respond to the task completion message by allowing the task completion message to remain on the individual sub-queue to keep the task container from executing another task routine from another request message on the group sub-queue, outputting a second request message onto the individual sub-queue to cause execution of a second task routine within the same task container to perform a second task, and responding to the second task in-progress message by de-queuing the task completion message. |
| Inventors: | SAS Institute Inc. (Cary, NC, US) |
| Assignees: | SAS Institute Inc. (Cary, NC, US) |
| Claim: | 1. An apparatus comprising at least one processor and a storage to store instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising: receive, at the at least one processor, and from a requesting device via a network, a request to perform a job flow comprising a set of tasks; within a performance container, the at least one processor is caused to output a first task execution request message onto a group sub-queue of a task queue to convey, to a set of task containers sharing access to the group sub-queue, a request to execute a first task routine to perform a first task of the set of tasks; within a first task container of the set of task containers, and in response to the output of the first task execution request message onto the group sub-queue, the at least one processor is caused to perform operations of the first task comprising: accede to executing the first task routine by outputting a first task in-progress message onto a first individual sub-queue of the task queue, wherein access to the first individual sub-queue is not shared with other task containers; execute the first task routine to generate at least one portion of a data object as part of performing the first task; store the at least one portion of the data object within at least one federated area; and output a first task completion message onto the first individual sub-queue of the task queue; and within the performance container, and in response to the output of the first task completion message onto the first individual sub-queue, the at least one processor is caused to perform operations comprising: determine, based on data dependencies among the set of tasks, whether a second task of the set of tasks uses the at least one portion of the data object as an input; in response to a determination that the second task uses the at least one portion of the data object as an input, perform operations comprising: while allowing the first task completion message to remain on the first individual sub-queue to cause the first task container to refrain from acceding to executing another task routine from another task routine execution request message on the group sub-queue, output a second task execution request message onto the first individual sub-queue to cause execution of a second task routine within the first task container to perform the second task using a buffered copy of the at least one portion of the data object as input; and in response to output of a second task in-progress message onto the first individual sub-queue from the first task container to accede to executing the second task routine, de-queue the first task completion message; and in response to a determination that the second task routine does not use the at least one portion of the data object as input, the at least one processor is caused to de-queue the first task completion message from the first individual sub-queue to enable the first task container to accede to executing another task routine from another task routine execution request message on the group sub-queue. |
| Claim: | 2. The apparatus of claim 1 , wherein: each task container of the set of task containers is of a first type that supports executions of multiple instances of task routines at least partially in parallel; the data object is generated in a distributed form as a set of data object blocks during executions of multiple instances of the first task routine across multiple task containers of the set of task containers, including the execution of the first task routine within the first task container; the at least one portion of the data object generated during the execution of the first task routine within the first task container comprises a first data object block of the set of data object blocks; within the performance container, the at least one processor is caused to output a third task execution request message onto the group sub-queue to convey, to the set of task containers, a request to execute the first task routine to perform the first task to generate a second data object block of the set of data object blocks; and within a second task container of the set of task containers, and in response to the output of the third task execution request message onto the group sub-queue, the at least one processor is caused to perform operations of the first task comprising: accede to executing the first task routine requested in the third task routine execution request message by outputting a third task in-progress message onto a second individual sub-queue of the task queue, wherein access to the second individual sub-queue is not shared with other task containers; and execute the first task routine to generate the second data object block as part of performing the first task. |
| Claim: | 3. The apparatus of claim 1 , wherein: each task container of the set of task containers is of a first type that supports executions of multiple instances of task routines at least partially in parallel; within the performance container, the at least one processor is caused to output a third task execution request message onto the group sub-queue to convey, to the set of task containers, a request to execute the first task routine to perform the first task to generate a second data object block of the set of data object blocks; within the first task container, and in response to the output of the second task execution request message onto the first individual sub-queue, the at least one processor is caused to perform operations of the second task comprising: accede to executing the second task routine by outputting the second task in-progress message onto the first individual sub-queue of the task queue; execute the second task routine using the first data object block as an input as part of performing the second task; and output a second task completion message onto the first individual sub-queue of the task queue; within the performance container, and in response to the output of the second task completion message onto the first individual sub-queue, the at least one processor is caused to perform operations comprising: determine, based on the data dependencies among the set of tasks, whether there is another task of the set of tasks that uses data output by the second task as an input; and in response to a determination that there is not another task that uses data output by the second task as an input, de-queue the second task completion message from the first individual sub-queue to enable the first task container to accede to executing another task routine from another task routine execution request message on the group sub-queue; and within the first task container, in response to the de-queuing of the second task completion message, and in response to the output of the third task execution request message onto the group sub-queue, the at least one processor is caused to perform further operations of the first task comprising: accede to executing the first task routine that is requested in the third task routine execution request message by outputting a third task in-progress message onto the first individual sub-queue; and execute the first task routine to generate the second data object block as part of performing the first task. |
| Claim: | 4. The apparatus of claim 1 , wherein: each task container of the set of task containers is of a first type that supports executions of multiple instances of task routines at least partially in parallel; the at least one processor executes instructions of a resource allocation routine to cause the at least one processor to dynamically allocate multiple containers based on availability of at least one of processing resources and storage resources; and within the performance container, and in response to commencement of performance of the first task, the at least one processor is caused to provide, to the resource allocation routine, an indication of at least one of a need for provision of more task containers of the first type or a need for provision of fewer task containers of a second type that supports executions of single instances of task routines. |
| Claim: | 5. The apparatus of claim 4 , wherein: the task queue is allocated to convey just messages associated with the execution of multiple instances of task routines by the first type of task container; and another task queue is allocated to convey just messages associated with the execution of single instances of task routines by a second type of task container. |
| Claim: | 6. The apparatus of claim 1 , wherein: data objects are stored within the at least one federated area in a format that is associated with syntax of a first programming language in which at least a subset of task routines are written; the first task container provides a memory space within the first task container to support exchanging a data object generated in a format that is associated with syntax of a second programming language between two task routines written in the second programming language; and within the first task container, and in response to the first task routine being written in the second programming language, the at least one processor is caused to perform operations comprising: convert the at least one portion of the first data object into the format associated with the syntax of the first programming language for storage within the at least one federated area, and for being buffered within the device in which the first task container is maintained; store another copy of the at least one portion of the first data object, as generated by execution of the first task routine in the format associated with the syntax of the second programming language, within the memory space; and in response to the first task container being caused to execute the second task routine immediately after the execution of the first task routine, and in response to the second task routine also being written in the second programming language, use the copy of the at least one portion of the first data object, as stored in the memory space, as an input to the second task routine. |
| Claim: | 7. The apparatus of claim 6 , wherein, within the first task container, in response to the first task routine being written in the second programming language, in response to the first task container being caused to execute the second task routine immediately after the execution of the first task routine, and in response to the second task routine being written in the first programming language, the at least one processor is caused to use the buffered copy of the at least one portion of the first data object as an input to the second task routine. |
| Claim: | 8. The apparatus of claim 1 , wherein: the job flow is defined in a job flow definition that specifies a set of tasks to be performed by executing a corresponding set of task routines, and that specifies data dependencies among the set of tasks; the set of tasks comprises the first task and the second task; the task queue comprises the group sub-queue, and a set of individual sub-queues; the set of individual sub-queues comprises the first individual sub-queue; and each individual sub-queue of the set of individual sub-queues is accessible to a different task container of the set of task containers to provide each task container of the set of task containers with a path of communication with the performance container that is not shared with any other task container. |
| Claim: | 9. The apparatus of claim 1 , wherein: the group sub-queue is maintained throughout at least the performance of the job flow; the first individual sub-queue is newly instantiated each time the first task container accedes to executing a task routine that is requested in a task routine execution request message that is output onto the group sub-queue; acceding to executing the first task routine comprises instantiating the first individual sub-queue before outputting the first task in-progress message onto the first individual sub-queue; and in response to the determination that the second task does not use the at least one portion of the first data object as an input, and in response to de-queuing of the first task completion message from the first individual sub-queue, the at least one processor is caused to uninstantiate the first individual sub-queue. |
| Claim: | 10. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause at least one processor to perform operations comprising: receive, at the at least one processor, and from a requesting device via a network, a request to perform a job flow comprising a set of tasks; within a performance container, the at least one processor is caused to output a first task execution request message onto a group sub-queue of a task queue to convey, to a set of task containers sharing access to the group sub-queue, a request to execute a first task routine to perform a first task of the set of tasks; within a first task container of the set of task containers, and in response to the output of the first task execution request message onto the group sub-queue, the at least one processor is caused to perform operations of the first task comprising: accede to executing the first task routine by outputting a first task in-progress message onto a first individual sub-queue of the task queue, wherein access to the first individual sub-queue is not shared with other task containers; execute the first task routine to generate at least one portion of a data object as part of performing the first task; store the at least one portion of the data object within at least one federated area; and output a first task completion message onto the first individual sub-queue of the task queue; and within the performance container, and in response to the output of the first task completion message onto the first individual sub-queue, the at least one processor is caused to perform operations comprising: determine, based on data dependencies among the set of tasks, whether a second task of the set of tasks uses the at least one portion of the data object as an input; in response to a determination that the second task uses the at least one portion of the data object as an input, perform operations comprising: while allowing the first task completion message to remain on the first individual sub-queue to cause the first task container to refrain from acceding to executing another task routine from another task routine execution request message on the group sub-queue, output a second task execution request message onto the first individual sub-queue to cause execution of a second task routine within the first task container to perform the second task using a buffered copy of the at least one portion of the data object as input; and in response to output of a second task in-progress message onto the first individual sub-queue from the first task container to accede to executing the second task routine, de-queue the first task completion message; and in response to a determination that the second task routine does not use the at least one portion of the data object as input, the at least one processor is caused to de-queue the first task completion message from the first individual sub-queue to enable the first task container to accede to executing another task routine from another task routine execution request message on the group sub-queue. |
| Claim: | 11. The computer-program product of claim 10 , wherein: each task container of the set of task containers is of a first type that supports executions of multiple instances of task routines at least partially in parallel; the data object is generated in a distributed form as a set of data object blocks during executions of multiple instances of the first task routine across multiple task containers of the set of task containers, including the execution of the first task routine within the first task container; the at least one portion of the data object generated during the execution of the first task routine within the first task container comprises a first data object block of the set of data object blocks; within the performance container, the at least one processor is caused to output a third task execution request message onto the group sub-queue to convey, to the set of task containers, a request to execute the first task routine to perform the first task to generate a second data object block of the set of data object blocks; and within a second task container of the set of task containers, and in response to the output of the third task execution request message onto the group sub-queue, the at least one processor is caused to perform operations of the first task comprising: accede to executing the first task routine requested in the third task routine execution request message by outputting a third task in-progress message onto a second individual sub-queue of the task queue, wherein access to the second individual sub-queue is not shared with other task containers; and execute the first task routine to generate the second data object block as part of performing the first task. |
| Claim: | 12. The computer-program product of claim 10 , wherein: each task container of the set of task containers is of a first type that supports executions of multiple instances of task routines at least partially in parallel; within the performance container, the at least one processor is caused to output a third task execution request message onto the group sub-queue to convey, to the set of task containers, a request to execute the first task routine to perform the first task to generate a second data object block of the set of data object blocks; within the first task container, and in response to the output of the second task execution request message onto the first individual sub-queue, the at least one processor is caused to perform operations of the second task comprising: accede to executing the second task routine by outputting the second task in-progress message onto the first individual sub-queue of the task queue; execute the second task routine using the first data object block as an input as part of performing the second task; and output a second task completion message onto the first individual sub-queue of the task queue; within the performance container, and in response to the output of the second task completion message onto the first individual sub-queue, the at least one processor is caused to perform operations comprising: determine, based on the data dependencies among the set of tasks, whether there is another task of the set of tasks that uses data output by the second task as an input; and in response to a determination that there is not another task that uses data output by the second task as an input, de-queue the second task completion message from the first individual sub-queue to enable the first task container to accede to executing another task routine from another task routine execution request message on the group sub-queue; and within the first task container, in response to the de-queuing of the second task completion message, and in response to the output of the third task execution request message onto the group sub-queue, the at least one processor is caused to perform further operations of the first task comprising: accede to executing the first task routine that is requested in the third task routine execution request message by outputting a third task in-progress message onto the first individual sub-queue; and execute the first task routine to generate the second data object block as part of performing the first task. |
| Claim: | 13. The computer-program product of claim 10 , wherein: each task container of the set of task containers is of a first type that supports executions of multiple instances of task routines at least partially in parallel; the at least one processor executes instructions of a resource allocation routine to cause the at least one processor to dynamically allocate multiple containers based on availability of at least one of processing resources and storage resources; and within the performance container, and in response to commencement of performance of the first task, the at least one processor is caused to provide, to the resource allocation routine, an indication of at least one of a need for provision of more task containers of the first type or a need for provision of fewer task containers of a second type that supports executions of single instances of task routines. |
| Claim: | 14. The computer-program product of claim 13 , wherein: the task queue is allocated to convey just messages associated with the execution of multiple instances of task routines by the first type of task container; and another task queue is allocated to convey just messages associated with the execution of single instances of task routines by a second type of task container. |
| Claim: | 15. The computer-program product of claim 10 , wherein: data objects are stored within the at least one federated area in a format that is associated with syntax of a first programming language in which at least a subset of task routines are written; the first task container provides a memory space within the first task container to support exchanging a data object generated in a format that is associated with syntax of a second programming language between two task routines written in the second programming language; and within the first task container, and in response to the first task routine being written in the second programming language, the at least one processor is caused to perform operations comprising: convert the at least one portion of the first data object into the format associated with the syntax of the first programming language for storage within the at least one federated area, and for being buffered within the device in which the first task container is maintained; store another copy of the at least one portion of the first data object, as generated by execution of the first task routine in the format associated with the syntax of the second programming language, within the memory space; and in response to the first task container being caused to execute the second task routine immediately after the execution of the first task routine, and in response to the second task routine also being written in the second programming language, use the copy of the at least one portion of the first data object, as stored in the memory space, as an input to the second task routine. |
| Claim: | 16. The computer-program product of claim 15 , wherein, within the first task container, in response to the first task routine being written in the second programming language, in response to the first task container being caused to execute the second task routine immediately after the execution of the first task routine, and in response to the second task routine being written in the first programming language, the at least one processor is caused to use the buffered copy of the at least one portion of the first data object as an input to the second task routine. |
| Claim: | 17. The computer-program product of claim 10 , wherein: the job flow is defined in a job flow definition that specifies a set of tasks to be performed by executing a corresponding set of task routines, and that specifies data dependencies among the set of tasks; the set of tasks comprises the first task and the second task; the task queue comprises the group sub-queue, and a set of individual sub-queues; the set of individual sub-queues comprises the first individual sub-queue; and each individual sub-queue of the set of individual sub-queues is accessible to a different task container of the set of task containers to provide each task container of the set of task containers with a path of communication with the performance container that is not shared with any other task container. |
| Claim: | 18. The computer-program product of claim 10 , wherein: the group sub-queue is maintained throughout at least the performance of the job flow; the first individual sub-queue is newly instantiated each time the first task container accedes to executing a task routine that is requested in a task routine execution request message that is output onto the group sub-queue; acceding to executing the first task routine comprises instantiating the first individual sub-queue before outputting the first task in-progress message onto the first individual sub-queue; and in response to the determination that the second task does not use the at least one portion of the first data object as an input, and in response to de-queuing of the first task completion message from the first individual sub-queue, the at least one processor is caused to uninstantiate the first individual sub-queue. |
| Claim: | 19. A computer-implemented method comprising: receiving, at the at least one processor, and from a requesting device via a network, a request to perform a job flow comprising a set of tasks; within a performance container, outputting a first task execution request message onto a group sub-queue of a task queue to convey, to a set of task containers sharing access to the group sub-queue, a request to execute a first task routine to perform a first task of the set of tasks; within a first task container of the set of task containers, and in response to the output of the first task execution request message onto the group sub-queue, performing operations of the first task comprising: acceding to executing the first task routine by outputting a first task in-progress message onto a first individual sub-queue of the task queue, wherein access to the first individual sub-queue is not shared with other task containers; executing, by the at least one processor, the first task routine to generate at least one portion of a data object as part of performing the first task; storing the at least one portion of the data object within at least one federated area; and outputting a first task completion message onto the first individual sub-queue of the task queue; and within the performance container, and in response to the output of the first task completion message onto the first individual sub-queue, performing operations comprising: determining, by the at least one processor, and based on data dependencies among the set of tasks, whether a second task of the set of tasks uses the at least one portion of the data object as an input; in response to a determination, by the at least one processor, that the second task uses the at least one portion of the data object as an input, performing operations comprising: while allowing the first task completion message to remain on the first individual sub-queue to cause the first task container to refrain from acceding to executing another task routine from another task routine execution request message on the group sub-queue, outputting a second task execution request message onto the first individual sub-queue to cause execution of a second task routine within the first task container to perform the second task using a buffered copy of the at least one portion of the data object as input; and in response to output of a second task in-progress message onto the first individual sub-queue from the first task container to accede to executing the second task routine, de-queuing the first task completion message; and in response to a determination, by the at least one processor, that the second task routine does not use the at least one portion of the data object as input, de-queuing the first task completion message from the first individual sub-queue to enable the first task container to accede to executing another task routine from another task routine execution request message on the group sub-queue. |
| Claim: | 20. The computer-implemented method of claim 19 , wherein: each task container of the set of task containers is of a first type that supports executions of multiple instances of task routines at least partially in parallel; the data object is generated in a distributed form as a set of data object blocks during executions of multiple instances of the first task routine across multiple task containers of the set of task containers, including the execution of the first task routine within the first task container; the at least one portion of the data object generated during the execution of the first task routine within the first task container comprises a first data object block of the set of data object blocks; and the method further comprises: within the performance container, outputting a third task execution request message onto the group sub-queue to convey, to the set of task containers, a request to execute the first task routine to perform the first task to generate a second data object block of the set of data object blocks; and within a second task container of the set of task containers, and in response to the output of the third task execution request message onto the group sub-queue, performing operations of the first task comprising: acceding to executing the first task routine requested in the third task routine execution request message by outputting a third task in-progress message onto a second individual sub-queue of the task queue, wherein access to the second individual sub-queue is not shared with other task containers; and executing, by the at least one processor, the first task routine to generate the second data object block as part of performing the first task. |
| Claim: | 21. The computer-implemented method of claim 19 , wherein: each task container of the set of task containers is of a first type that supports executions of multiple instances of task routines at least partially in parallel; and the method further comprises: within the performance container, outputting a third task execution request message onto the group sub-queue to convey, to the set of task containers, a request to execute the first task routine to perform the first task to generate a second data object block of the set of data object blocks; within the first task container, and in response to the output of the second task execution request message onto the first individual sub-queue, performing operations of the second task comprising: acceding to executing the second task routine by outputting the second task in-progress message onto the first individual sub-queue of the task queue; executing, by the at least one processor, the second task routine using the first data object block as an input as part of performing the second task; and outputting a second task completion message onto the first individual sub-queue of the task queue; within the performance container, and in response to the output of the second task completion message onto the first individual sub-queue, performing operations comprising: determining, by the at least one processor, and based on the data dependencies among the set of tasks, whether there is another task of the set of tasks that uses data output by the second task as an input; and in response to a determination, by the at least one processor, that there is not another task that uses data output by the second task as an input, de-queuing the second task completion message from the first individual sub-queue to enable the first task container to accede to executing another task routine from another task routine execution request message on the group sub-queue; and within the first task container, in response to the de-queuing of the second task completion message, and in response to the output of the third task execution request message onto the group sub-queue, performing further operations of the first task comprising: acceding to executing the first task routine that is requested in the third task routine execution request message by outputting a third task in-progress message onto the first individual sub-queue; and executing, by the at least one processor, the first task routine to generate the second data object block as part of performing the first task. |
| Claim: | 22. The computer-implemented method of claim 19 , wherein: each task container of the set of task containers is of a first type that supports executions of multiple instances of task routines at least partially in parallel; the at least one processor executes instructions of a resource allocation routine to cause the at least one processor to dynamically allocate multiple containers based on availability of at least one of processing resources and storage resources; and the method further comprises, within the performance container, and in response to commencement of performance of the first task, providing, to the resource allocation routine, an indication of at least one of a need for provision of more task containers of the first type or a need for provision of fewer task containers of a second type that supports executions of single instances of task routines. |
| Claim: | 23. The computer-implemented method of claim 22 , wherein: the task queue is allocated to convey just messages associated with the execution of multiple instances of task routines by the first type of task container; and another task queue is allocated to convey just messages associated with the execution of single instances of task routines by a second type of task container. |
| Claim: | 24. The computer-implemented method of claim 19 , wherein: data objects are stored within the at least one federated area in a format that is associated with syntax of a first programming language in which at least a subset of task routines are written; the first task container provides a memory space within the first task container to support exchanging a data object generated in a format that is associated with syntax of a second programming language between two task routines written in the second programming language; and the method further comprises, within the first task container, and in response to the first task routine being written in the second programming language, performing operations comprising: converting, by the at least one processor, the at least one portion of the first data object into the format associated with the syntax of the first programming language for storage within the at least one federated area, and for being buffered within the device in which the first task container is maintained; storing another copy of the at least one portion of the first data object, as generated by execution of the first task routine in the format associated with the syntax of the second programming language, within the memory space; and in response to the first task container being caused to execute the second task routine immediately after the execution of the first task routine, and in response to the second task routine also being written in the second programming language, using, by the at least one processor the copy of the at least one portion of the first data object, as stored in the memory space, as an input to the second task routine. |
| Claim: | 25. The computer-implemented method of claim 24 , comprising, within the first task container, in response to the first task routine being written in the second programming language, in response to the first task container being caused to execute the second task routine immediately after the execution of the first task routine, and in response to the second task routine being written in the first programming language, using the buffered copy of the at least one portion of the first data object as an input to the second task routine. |
| Claim: | 26. The computer-implemented method of claim 19 , wherein: the job flow is defined in a job flow definition that specifies a set of tasks to be performed by executing a corresponding set of task routines, and that specifies data dependencies among the set of tasks; the set of tasks comprises the first task and the second task; the task queue comprises the group sub-queue, and a set of individual sub-queues; the set of individual sub-queues comprises the first individual sub-queue; and each individual sub-queue of the set of individual sub-queues is accessible to a different task container of the set of task containers to provide each task container of the set of task containers with a path of communication with the performance container that is not shared with any other task container. |
| Claim: | 27. The computer-implemented method of claim 19 , wherein: the group sub-queue is maintained throughout at least the performance of the job flow; the first individual sub-queue is newly instantiated each time the first task container accedes to executing a task routine that is requested in a task routine execution request message that is output onto the group sub-queue; acceding to executing the first task routine comprises instantiating the first individual sub-queue before outputting the first task in-progress message onto the first individual sub-queue; and the method further comprises, in response to the determination that the second task does not use the at least one portion of the first data object as an input, and in response to de-queuing of the first task completion message from the first individual sub-queue, uninstantiating the first individual sub-queue. |
| Patent References Cited: | 9313133 April 2016 Yeddanapudi 9454323 September 2016 Dausner 9577972 February 2017 Word 9760376 September 2017 Bequet 9946719 April 2018 Bowman 9984004 May 2018 Little 9998418 June 2018 Clark 10042886 August 2018 Saadat-Panah 10169121 January 2019 Vibhor 10282690 May 2019 DelFranco 10360053 July 2019 Christensen 10361919 July 2019 Yang 10635642 April 2020 Haggerty 10691501 June 2020 Hussain 11144363 October 2021 Francis Conde 11481245 October 2022 Oliver 20060029068 February 2006 Frank 20130290979 October 2013 Kawano 20130332612 December 2013 Cai 20140040905 February 2014 Tsunoda 20150067028 March 2015 Kumar 20150149745 May 2015 Eble 20150205633 July 2015 Kaptur 20160371122 December 2016 Nair 20170093988 March 2017 Rehaag 20170255886 September 2017 Schmidt 20200133728 April 2020 Nataraj |
| Other References: | Nguyen et al.; “KOHA: Building a Kafka-based Distributed Queue System on the fly in a Hadoop cluster”; IEEE 2016; (Nguyen_2016.pdf; pp. 1-6) (Year: 2016). cited by examiner Yildiz et al.; “Fault-Tolerance in Dataflow-based Scientific Workflow Management”; 2010 IEEE 6th World Congress on Services; ( Yildiz_201 0.pdf; pp. 336-343) (Year: 2010). cited by applicant |
| Primary Examiner: | Patel, Hiren P |
| Attorney, Agent or Firm: | KDW FIRM PLLC |
| Dokumentencode: | edspgr.11762689 |
| Datenbank: | USPTO Patent Grants |
| Abstract: | An apparatus including a processor to: output a first request message onto a group sub-queue shared by multiple task containers to request execution of a first task routine; within a task container, respond to the first request message, by outputting a first task in-progress message onto an individual sub-queue not shared with other task containers to accede to executing the first task routine, followed by a task completion message; and respond to the task completion message by allowing the task completion message to remain on the individual sub-queue to keep the task container from executing another task routine from another request message on the group sub-queue, outputting a second request message onto the individual sub-queue to cause execution of a second task routine within the same task container to perform a second task, and responding to the second task in-progress message by de-queuing the task completion message. |
|---|