Artificial intelligence information processing device and artificial intelligence information processing method
Uloženo v:
| Název: | Artificial intelligence information processing device and artificial intelligence information processing method |
|---|---|
| Patent Number: | 12184,931 |
| Datum vydání: | December 31, 2024 |
| Appl. No: | 17/610476 |
| Application Filed: | March 09, 2020 |
| Abstrakt: | An artificial intelligence information processing device that generates information about a scene according to artificial intelligence is provided. The artificial intelligence information processing device includes a gaze degree estimation unit configured to estimate a degree of gaze of a user who is watching content according to artificial intelligence on the basis of sensor information, an acquisition unit configured to acquire a video of a scene at which the user gazes in the content and information about the content on the basis of an estimation result of the gaze degree estimation unit, and a scene information estimation unit configured to estimate information about the scene at which the user gazes according to artificial intelligence on the basis of the video of the scene at which the user gazes and the information about the content. |
| Inventors: | SONY GROUP CORPORATION (Tokyo, JP) |
| Assignees: | SONY GROUP CORPORATION (Tokyo, JP) |
| Claim: | 1. An artificial intelligence information processing device, comprising: circuitry configured to: receive sensor information of a user who watches first content; apply a first artificial intelligence operation on the received sensor information; estimate a degree of gaze of the user based on the applied first artificial intelligence operation; acquire, based on the estimated degree of gaze of the user, a video of a scene at which the user gazes in the first content and first information about the first content; apply a second artificial intelligence operation on the video of the scene and the first information about the first content; estimate second information about the scene based on the applied second artificial intelligence operation, wherein the second information about the scene includes first metadata that characterizes the scene; customize the first metadata-characterizing the scene based on an input of the user; acquire, based on the scene and the customized first metadata, a plurality of pieces of second content and second metadata corresponding to each piece of second content of the plurality of pieces of second content; and control a display device to display the second metadata and representative images of the plurality of pieces of second content. |
| Claim: | 2. The artificial intelligence information processing device according to claim 1 , wherein the circuitry is further configured to: estimate third information about a plurality of scenes based on the second artificial intelligence operation; and estimate, based on a neural network, the second information about the scene, the neural network is trained on correlations of a plurality of videos of the plurality of scenes and the third information about the plurality of scenes, the plurality of scenes includes the scene, and the plurality of videos includes the video. |
| Claim: | 3. The artificial intelligence information processing device according to claim 1 , wherein the circuitry is further configured to: estimate a plurality of degrees of gaze of the user based on the first artificial intelligence operation; and estimate, based on a neural network, the degree of gaze having a correlation with the sensor information, the neural network is trained on correlations between the sensor information and the plurality of degrees of gaze of the user, and the plurality of degrees of gaze includes the degree of gaze. |
| Claim: | 4. The artificial intelligence information processing device according to claim 3 , wherein the sensor information includes an image of the user who watches the first content, the circuitry is further configured to estimate, based on the neural network, the degree of gaze having a correlation with a face recognition result of a plurality of face recognition results in the image, and the neural network is trained on correlations between the plurality of face recognition results and the plurality of degrees of gaze of the user. |
| Claim: | 5. The artificial intelligence information processing device according to claim 1 , wherein the first content includes at least one of broadcast content or stream distribution content. |
| Claim: | 6. The artificial intelligence information processing device according to claim 1 , wherein the circuitry is further configured to control the display device to display the second information about the scene. |
| Claim: | 7. The artificial intelligence information processing device according to claim 1 , wherein the circuitry is further configured to control the display device to display the scene and the second information about the scene. |
| Claim: | 8. An artificial intelligence information processing device, comprising: circuitry configured to: receive sensor information of a user who watches first content; estimate, based on a first neural network, first information about a scene of a plurality of scenes in the first content at which the user gazes, wherein the first neural network is trained on correlations between the sensor information, the first content and second information on the first content, and third information about the plurality of scenes at which the user gazes, and the first information about the scene includes first metadata that characterizes the scene; customize the first metadata based on an input of the user; acquire, based on the scene and the customized first metadata, a plurality of pieces of second content and second metadata corresponding to each piece of second content of the plurality of pieces of second content; and control a display device to display the second metadata and representative images of the plurality of pieces of second content. |
| Claim: | 9. The artificial intelligence information processing device according to claim 8 , wherein the sensor information includes at least an image of the user who watches is the first content, the circuitry is further configured to estimate, based on a second neural network, the first information about thescene, and the second neural network is trained on correlations between face images, the first content and the second information on the first content, and the third information about the plurality of scenes at which the user gazes. |
| Claim: | 10. An artificial intelligence information processing method, comprising: receiving sensor information of a user who watches first content; applying a first artificial intelligence operation on the received sensor information; estimating a degree of gaze of the user based on the applied first artificial intelligence operation; acquiring, based on the estimated degree of gaze of the user, a video of a scene at which the user gazes in the first content and first information about the first content; applying a second artificial intelligence operation on the video of the scene and the first information about the first content; estimating second information about the scene based on the applied second artificial intelligence operation, wherein the second information about the scene includes first metadata that characterizes the scene; customizing the first metadata based on an input of the user; acquiring, based on the scene and the customized first metadata, a plurality of pieces of second content and second metadata corresponding to each piece of second content of the plurality of pieces of second content; and controlling a display device to display the second metadata and representative images of the plurality of pieces of second content. |
| Patent References Cited: | 10306311 May 2019 Knas 11353955 June 2022 Burgess 2007/0300258 December 2007 O'Connor 2012/0146891 June 2012 Kalinli 2013/0290904 October 2013 Hinman 2018/0007431 January 2018 Sidhu 2018/0288477 October 2018 Gupta 2018/0300751 October 2018 Hammitt 2018/0357317 December 2018 Santiago 2020/0045363 February 2020 Nellore 2020/0086879 March 2020 Lakshmi Narayanan 2020/0273485 August 2020 Jagmag 2020/0275158 August 2020 Gaur 2005-142975 June 2005 2007-143010 June 2007 2008-236779 October 2008 2011-239158 November 2011 4840393 December 2011 2018-205819 December 2018 2017/120469 July 2017 |
| Other References: | International Search Report and Written Opinion of PCT Application No. PCT/JP2020/009957, issued on May 26, 2020, 08 pages of ISRWO. cited by applicant |
| Primary Examiner: | Huerta, Alexander Q |
| Attorney, Agent or Firm: | CHIP LAW GROUP |
| Přístupové číslo: | edspgr.12184931 |
| Databáze: | USPTO Patent Grants |
| Abstrakt: | An artificial intelligence information processing device that generates information about a scene according to artificial intelligence is provided. The artificial intelligence information processing device includes a gaze degree estimation unit configured to estimate a degree of gaze of a user who is watching content according to artificial intelligence on the basis of sensor information, an acquisition unit configured to acquire a video of a scene at which the user gazes in the content and information about the content on the basis of an estimation result of the gaze degree estimation unit, and a scene information estimation unit configured to estimate information about the scene at which the user gazes according to artificial intelligence on the basis of the video of the scene at which the user gazes and the information about the content. |
|---|