Revisiting wavelet compression for large-scale climate data using JPEG 2000 and ensuring data precision

We revisit wavelet compression by using a standards-based method to reduce large-scale data sizes for production scientific computing. Many of the bottlenecks in visualization and analysis come from limited bandwidth in data movement, from storage to networks. The majority of the processing time for...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2011 IEEE Symposium on Large Data Analysis and Visualization s. 31 - 38
Hlavní autoři: Woodring, J., Mniszewski, S., Brislawn, C., DeMarle, D., Ahrens, J.
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.10.2011
Témata:
Bit rate
Data visualization
E.4 [Data]: Coding and Information Theory-Data compaction and compression
H.3.m [Information Storage and Retrieval]: Miscellaneous
I.3.8 [Computer Graphics]: Applications
Image coding
Quantization
Transform coding
Wavelet transforms
ISBN:9781467301565, 1467301566
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We revisit wavelet compression by using a standards-based method to reduce large-scale data sizes for production scientific computing. Many of the bottlenecks in visualization and analysis come from limited bandwidth in data movement, from storage to networks. The majority of the processing time for visualization and analysis is spent reading or writing large-scale data or moving data from a remote site in a distance scenario. Using wavelet compression in JPEG 2000, we provide a mechanism to vary data transfer time versus data quality, so that a domain expert can improve data transfer time while quantifying compression effects on their data. By using a standards-based method, we are able to provide scientists with the state-of-the-art wavelet compression from the signal processing and data compression community, suitable for use in a production computing environment. To quantify compression effects, we focus on measuring bit rate versus maximum error as a quality metric to provide precision guarantees for scientific analysis on remotely compressed POP (Parallel Ocean Program) data.
ISBN:9781467301565
1467301566
DOI:10.1109/LDAV.2011.6092314