Coding of data streams in a vast storage network
Gespeichert in:
| Titel: | Coding of data streams in a vast storage network |
|---|---|
| Patent Number: | 11907,060 |
| Publikationsdatum: | February 20, 2024 |
| Appl. No: | 17/663299 |
| Application Filed: | May 13, 2022 |
| Abstract: | A method begins by a processing module concurrently receiving a first data stream and a second data stream for transmission to a receiving entity. The method continues with the processing module dividing each of the first and second data streams to produce a first plurality of data blocks corresponding to the first data stream and a second plurality of data blocks corresponding to the second data stream, where data blocks of the first plurality of data blocks are time aligned with data blocks of the second plurality of data blocks. The method continues with the processing module creating a data matrix from the first and second plurality of data blocks and generating a coded matrix from the data matrix and an encoding matrix. The method continues with the processing module outputting a plurality of pairs of coded values of the coded matrix to the receiving entity. |
| Inventors: | Pure Storage, Inc. (Mountain View, CA, US) |
| Assignees: | Pure Storage, Inc. (Santa Clara, CA, US) |
| Claim: | 1. A method comprises: concurrently receiving a first data stream and a second data stream for transmission to a receiving entity; dividing each of the first and second data streams to produce a first plurality of data blocks corresponding to the first data stream and a second plurality of data blocks corresponding to the second data stream, wherein at least one block of the first plurality of data blocks is time aligned with at least one block of the second plurality of data blocks; creating a data matrix from the first and second plurality of data blocks; generating a coded matrix by multiplying the data matrix with an encoding matrix; and outputting a plurality of pairs of coded values of the coded matrix to the receiving entity, wherein a pair of coded values includes a first coded value corresponding to the one of the first plurality of data blocks and a second coded value corresponding to the one of the second plurality of data blocks. |
| Claim: | 2. The method of claim 1 , wherein outputting the plurality of pairs of coded values comprises: outputting pairs of coded values of the coded matrix in a sequential order corresponding to a time ordering of the first and second plurality of data blocks the pairs of coded values. |
| Claim: | 3. The method of claim 1 , wherein outputting the plurality of pairs of coded values comprises: outputting a decode threshold number of pairs of coded values of the coded matrix such that coded values of the decode threshold number of pairs of coded values associated with a first plurality of data blocks can be decoded to recapture each data block of the first plurality of data blocks and such that coded values of the decode threshold number of pairs of coded values associated with a second plurality of data blocks can be decoded to recapture each data block of the second plurality of data blocks. |
| Claim: | 4. The method of claim 1 , wherein outputting the one or more pairs of coded values further comprises: receiving a request for one or more additional pairs of coded values from the receiving entity; and outputting the one or more additional pairs of coded values to the receiving entity when the one or more additional pairs of coded values are available. |
| Claim: | 5. The method of claim 1 , wherein the encoding matrix comprises at least one of: a Reed-Solomon based encoding matrix, an on-line coding based matrix, a Cauchy Reed-Solomon based encoding matrix, a forward error correction based matrix, and an erasure code based matrix. |
| Claim: | 6. The method of claim 1 further comprises: locally storing the coded matrix for a period of time T. |
| Claim: | 7. The method of claim 6 further comprises: using the locally stored coded matrix to facilitate subsequent transmission of coded values to the receiving entity. |
| Claim: | 8. The method of claim 1 further comprises: concurrently receiving a third data stream with the first and second data streams; dividing each of the first, second and third data streams to produce the first plurality of data blocks corresponding to the first data stream, a second plurality of data blocks corresponding to the second data stream, and a third plurality of data blocks corresponding to the third data stream, wherein at least one block each of the first plurality of data blocks, the second plurality of data blocks and the third plurality of data blocks are time aligned; creating the data matrix from the first, second, and third plurality of data blocks; generating the coded matrix from the data matrix and the encoding matrix; and outputting one or more trios of coded values of the coded matrix to the receiving entity, wherein a trio of coded values of the one or more trios of coded values includes the coded value corresponding to the one of the first plurality of data blocks, the coded value corresponding to the one of the second plurality of data blocks, and a coded value corresponding to the one of the third plurality of data blocks. |
| Claim: | 9. The method of claim 1 , wherein the creating the data matrix comprises: placing first time corresponding data blocks of the first and second plurality of data blocks into a first row of the data matrix; and placing second time corresponding data blocks of the first and second plurality of data blocks into a second row of the data matrix. |
| Claim: | 10. The method of claim 1 further comprises: the first data stream corresponding to a first recording of an environment; and the second data stream corresponding to a second recording of the environment. |
| Claim: | 11. A storage network module comprises: a first module, when operable within a computing device, causes the computing device to: concurrently receive a first data stream and a second data stream for transmission to a receiving entity; a second module, when operable within the computing device, causes the computing device to: divide each of the first and second data streams to produce a first plurality of data blocks corresponding to the first data stream and a second plurality of data blocks corresponding to the second data stream, wherein at least one block of the first plurality of data blocks is time aligned with at least one block of the second plurality of data blocks; create a data matrix from the first and second plurality of data blocks; a third module, when operable within the computing device, causes the computing device to: generate a coded matrix by multiplying the data matrix with an encoding matrix; and a fourth module, when operable within the computing device, causes the computing device to: output a plurality of pairs of coded values of the coded matrix to the receiving entity, wherein a pair of coded values includes a first coded value corresponding to the one of the first plurality of data blocks and a second coded value corresponding to the one of the second plurality of data blocks. |
| Claim: | 12. The storage module of claim 11 , wherein the fourth module functions to output the one or more pairs of coded values by: outputting pairs of coded values of the coded matrix in a sequential order corresponding to a time ordering of the first and second plurality of data blocks the pairs of coded values. |
| Claim: | 13. The storage module of claim 11 , wherein the fourth module further functions to output the one or more pairs of coded values by: outputting a decode threshold number of pairs of coded values of the coded matrix such that coded values of the decode threshold number of pairs of coded values associated with a first plurality of data blocks can be decoded to recapture each data block of the first plurality of data blocks and such that coded values of the decode threshold number of pairs of coded values associated with a second plurality of data blocks can be decoded to recapture each data block of the second plurality of data blocks. |
| Claim: | 14. The storage module of claim 11 , wherein the fourth module further functions to output the one or more pairs of coded values by: receiving a request for one or more additional pairs of coded values from the receiving entity; and outputting the one or more additional pairs of coded values to the receiving entity when the one or more additional pairs of coded values are available. |
| Claim: | 15. The storage module of claim 11 , wherein the encoding matrix comprises at least one of: a Reed-Solomon based encoding matrix, an on-line coding based matrix, a Cauchy Reed-Solomon based encoding matrix, a forward error correction based matrix, and an erasure code based matrix. |
| Claim: | 16. The storage module of claim 11 , wherein the third module further functions to locally store the coded matrix for a period of time T. |
| Claim: | 17. The storage module of claim 16 , wherein the third module further functions to use the locally stored coded matrix to facilitate subsequent transmission of coded values to the receiving entity. |
| Claim: | 18. The storage module of claim 11 , wherein: the first module further functions to concurrently receive a third data stream with the first and second data streams; the second module further functions to divide each of the first, second, and third data streams to produce the first plurality of data blocks, the second plurality of blocks, and a third plurality of data blocks corresponding to the third data stream, wherein the one of the third plurality of data blocks is time aligned with the one of the first plurality of data blocks and with the one of the second plurality of data blocks and create the data matrix from the first, second, and third plurality of data blocks; the third module further functions to generate the coded matrix from the data matrix and the encoding matrix; and the fourth module further functions to output one or more trios of coded values of the coded matrix to the receiving entity, wherein a trio of coded values of the one or more trios of coded values includes the coded value corresponding to the one of the first plurality of data blocks, the coded value corresponding to the one of the second plurality of data blocks, and a coded value corresponding to the one of the third plurality of data blocks. |
| Claim: | 19. The storage module of claim 11 , wherein the second module functions to create the data matrix by: placing first time corresponding data blocks of the first and second plurality of data blocks into a first row of the data matrix; and placing second time corresponding data blocks of the first and second plurality of data blocks into a second row of the data matrix. |
| Claim: | 20. The storage module of claim 11 , wherein the first data stream corresponds to a first recording of an environment and the second data stream corresponds to a second recording of the environment. |
| Patent References Cited: | 4092732 May 1978 Ouchi 5454101 September 1995 Mackay 5485474 January 1996 Rabin 5774643 June 1998 Lubbers 5809285 September 1998 Hilland 5890156 March 1999 Rekieta 5987622 November 1999 Lo Verso 5991414 November 1999 Garay 6012159 January 2000 Fischer 6058454 May 2000 Gerlach 6070003 May 2000 Gove et al. 6128277 October 2000 Bruck 6175571 January 2001 Haddock 6192472 February 2001 Garay 6256688 July 2001 Suetaka 6272658 August 2001 Steele 6301604 October 2001 Nojima 6356949 March 2002 Katsandres 6366995 April 2002 Nikolaevich 6374336 April 2002 Peters 6415373 July 2002 Peters 6418539 July 2002 Walker 6567948 May 2003 Steele 6571282 May 2003 Bowman-Amuah 6609223 August 2003 Wolfgang 6718361 April 2004 Basani 6760808 July 2004 Peters 6785783 August 2004 Buckland 6826711 November 2004 Moulton 6879596 April 2005 Dooply 7003688 February 2006 Pittelkow 7024451 April 2006 Jorgenson 7024609 April 2006 Wolfgang 7103824 September 2006 Halford 7103915 September 2006 Redlich 7140044 November 2006 Redlich 7146644 December 2006 Redlich 7171493 January 2007 Shu 7222133 May 2007 Raipurkar 7240236 July 2007 Cutts 7636724 December 2009 De La Torre 8532212 September 2013 Ito 8751894 June 2014 Grube 8930649 January 2015 Grube 9213742 December 2015 Grube 9715425 July 2017 Grube 10235237 March 2019 Grube 10298683 May 2019 Litvinsky 11334425 May 2022 Grube 20020062422 May 2002 Butterworth 20020166079 November 2002 Ulrich 20030018927 January 2003 Gadir 20030037261 February 2003 Meffert 20030065617 April 2003 Watkins 20030084020 May 2003 Shu 20040024963 February 2004 Talagala 20040122917 June 2004 Menon 20040215998 October 2004 Buxton 20040228493 November 2004 Ma 20050100022 May 2005 Ramprashad 20050114594 May 2005 Corbett 20050125593 June 2005 Karpoff 20050131993 June 2005 Fatula 20050132070 June 2005 Redlich 20050144382 June 2005 Schmisseur 20050229069 October 2005 Hassner 20060047907 March 2006 Shiga 20060105724 May 2006 Nakao 20060136448 June 2006 Cialini 20060147219 July 2006 Yoshino et al. 20060156059 July 2006 Kitamura 20060224603 October 2006 Correll 20070079081 April 2007 Gladwin 20070079082 April 2007 Gladwin 20070079083 April 2007 Gladwin 20070088970 April 2007 Buxton 20070174192 July 2007 Gladwin 20070214285 September 2007 Au 20070234110 October 2007 Soran 20070283167 December 2007 Venters, III 20090094251 April 2009 Gladwin 20090094318 April 2009 Gladwin 20100023524 January 2010 Gladwin 20100091842 April 2010 Ikeda et al. 20100180176 July 2010 Yosoku et al. 20100287200 November 2010 Dhuse 20110053639 March 2011 Etienne Suanez et al. 20120051208 March 2012 Li et al. 20120106595 May 2012 Bhattad et al. |
| Other References: | Chung; An Automatic Data Segmentation Method for 3D Measured Data Points; National Taiwan University, pp. 1-8; 1998. cited by applicant Harrison; Lightweight Directory Access Protocol (LDAP): Authentication Methods and Security Mechanisms; IETF Network Working Group; RFC 4513; Jun. 2006; pp. 1-32. cited by applicant Kubiatowicz, et al.; OceanStore: An Architecture for Global-Scale Persistent Storage; Proceedings of the Ninth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2000); Nov. 2000; pp. 1-12. cited by applicant Legg; Lightweight Directory Access Protocol (LDAP): Syntaxes and Matching Rules; IETF Network Working Group; RFC 4517; Jun. 2006; pp. 1-50. cited by applicant Plank, T1: Erasure Codes for Storage Applications; FAST2005, 4th Usenix Conference on File Storage Technologies; Dec. 13-16, 2005; pp. 1-74. cited by applicant Rabin; Efficient Dispersal of Information for Security, Load Balancing, and Fault Tolerance; Journal of the Association for Computer Machinery; vol. 36, No. 2; Apr. 1989; pp. 335-348. cited by applicant Satran, et al.; Internet Small Computer Systems Interface (ISCSI); Ietf Network Working Group; RFC 3720; Apr. 2004; pp. 1-257. cited by applicant Sciberras; Lightweight Directory Access Protocol (LDAP): Schema for User Applications; IETF Network Working Group; RFC 4519; Jun. 2006; pp. 1-33. cited by applicant Sermersheim; Lightweight Directory Access Protocol (LDAP): The Protocol; IETF Network Working Group; RFC 4511; Jun. 2006; pp. 1-68. cited by applicant Shamir; How to Share a Secret; Communications of the ACM; vol. 22, No. 11; Nov. 1979; pp. 612-613. cited by applicant Smith; Lightweight Directory Access Protocol (LDAP): Uniform Resource Locator; IETF Network Working Group; RFC 4516; Jun. 2006; pp. 1-15. cited by applicant Smith; Lightweight Directory Access Protocol (LDAP): String Representation of Search Filters; IETF Network Working Group; RFC 4515; Jun. 2006; pp. 1-12. cited by applicant Wildi; Java iSCSi Initiator; Master Thesis; Department of Computer and Information Science, University of Konstanz; Feb. 2007; 60 pgs. cited by applicant Xin, et al.; Evaluation of Distributed Recovery in Large-Scale Storage Systems; 13th IEEE International Symposium on High Performance Distributed Computing; Jun. 2004; pp. 172-181. cited by applicant Zeilenga; Lightweight Directory Access Protocol (LDAP): Directory Information Models; IETF Network Working Group; RFC 4512; Jun. 2006; pp. 1-49. cited by applicant Zeilenga; Lightweight Directory Access Protocol (LDAP): Internationalized String Preparation; IETF Network Working Group; RFC 4518; Jun. 2006; pp. 1-14. cited by applicant Zeilenga; Lightweight Directory Access Protocol (LDAP): String Representation of Distinguished Names; IETF Network Working Group; RFC 4514; Jun. 2006; pp. 1-15. cited by applicant Zeilenga; Lightweight Directory Access Protocol (LDAP): Technical Specification Road Map; IETF Network Working Group; RFC 4510; Jun. 2006; pp. 1-8. cited by applicant |
| Primary Examiner: | Rizk, Samir W |
| Attorney, Agent or Firm: | Garlick & Markison Markison, Timothy W. Hale, Kelly H. |
| Dokumentencode: | edspgr.11907060 |
| Datenbank: | USPTO Patent Grants |
| Abstract: | A method begins by a processing module concurrently receiving a first data stream and a second data stream for transmission to a receiving entity. The method continues with the processing module dividing each of the first and second data streams to produce a first plurality of data blocks corresponding to the first data stream and a second plurality of data blocks corresponding to the second data stream, where data blocks of the first plurality of data blocks are time aligned with data blocks of the second plurality of data blocks. The method continues with the processing module creating a data matrix from the first and second plurality of data blocks and generating a coded matrix from the data matrix and an encoding matrix. The method continues with the processing module outputting a plurality of pairs of coded values of the coded matrix to the receiving entity. |
|---|