Advanced Python Programming Accelerate your Python programs using proven techniques and design patterns
Write fast, robust, and highly reusable applications using Python’s internal optimization, state-of-the-art performance-benchmarking tools, and cutting-edge libraries Key Features Benchmark, profile, and accelerate Python programs using optimization toolsScale applications to multiple processors wit...
Uloženo v:
| Hlavní autor: | |
|---|---|
| Médium: | E-kniha |
| Jazyk: | angličtina |
| Vydáno: |
Birmingham
Packt Publishing
2022
Packt Publishing, Limited Packt Publishing Limited |
| Vydání: | 2 |
| Témata: | |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
Obsah:
- Table of Contents Benchmarking and ProfilingPure Python OptimizationsFast Array Operations with NumPy and PandasC Performance with CythonExploring CompilersAutomatic Differentiation and Accelerated Linear Algebra for Machine LearningImplementing ConcurrencyParallel ProcessingConcurrent Web RequestsConcurrent Image ProcessingBuilding Communication Channels with asyncioDeadlocksStarvationRace ConditionsThe Global Interpreter LockThe Factory PatternThe Builder PatternOther Creational PatternsThe Adapter PatternThe Decorator PatternThe Bridge PatternThe Façade PatternOther Structural PatternsThe Chain of Responsibility PatternThe Command PatternThe Observer Pattern
- Cover -- Title Page -- Copyright and credits -- Contributors -- About the reviewers -- Table of Contents -- Preface -- Section 1: Python-Native and Specialized Optimization -- Chapter 1: Benchmarking and Profiling -- Technical requirements -- Designing your application -- Building a particle simulator -- Visualizing the simulation -- Writing tests and benchmarks -- Timing your benchmark -- Writing better tests and benchmarks with pytest-benchmark -- Finding bottlenecks with cProfile -- Graphically analyzing profiling results -- Profiling line by line with line_profiler -- Optimizing our code -- Using the dis module -- Profiling memory usage with memory_profiler -- Summary -- Questions -- Further reading -- Chapter 2: Pure Python Optimizations -- Technical requirements -- Using the right algorithms and data structures -- Lists and deques -- Dictionaries -- Sets -- Heaps -- Tries -- Improved efficiency with caching and memoization -- Joblib -- Efficient iteration with comprehensions and generators -- Summary -- Questions -- Further reading -- Chapter 3: Fast ArrayOperations withNumPy, Pandas,and Xarray -- Technical requirement -- Getting started with NumPy -- Creating arrays -- Accessing arrays -- Broadcasting -- Mathematical operations -- Calculating the norm -- Rewriting the particle simulator in NumPy -- Reaching optimal performance with numexpr -- Working with database-style data with pandas -- pandas fundamentals -- Database-style operations with pandas -- High-performance labeled data with xarray -- Analyzing concentration -- The xarray library -- Improved performance -- Plotting with xarray -- Chapter 4: C Performance with Cython -- Technical requirements -- Compiling Cython extensions -- Adding static types -- Declaring variables -- Declaring functions -- Declaring classes -- Sharing declarations -- Working with arrays
- C arrays and pointers -- Working with NumPy arrays -- Working with typed memoryviews -- Using a particle simulator in Cython -- Profiling Cython -- Using Cython with Jupyter -- Summary -- Questions -- Chapter 5: Exploring Compilers -- Technical requirements -- Getting started with Numba -- Using Numba decorators -- Type specializations -- Object mode versus native mode -- Numba and NumPy -- JIT classes -- Limitations in Numba -- The PyPy project -- Setting up PyPy -- Running a particle simulator in PyPy -- Other interesting projects -- Summary -- Questions -- Further reading -- Chapter 6: Automatic Differentiation and Accelerated Linear Algebra for Machine Learning -- A crash course in machine learning -- Model parameters -- Loss function -- Loss minimization -- Getting JAX up and running -- Installing JAX -- Using Google Colab -- Automatic differentiation for loss minimization -- Making the dataset -- Building a linear model -- Gradient descent with automatic differentiation -- Just-In-Time compilation for improved efficiency -- Automatic vectorization for efficient kernels -- Data that is not linearly separable -- The kernel method in machine learning -- Automatic vectorization for kernelized models -- Summary -- Questions -- Further reading -- Section 2: Concurrency and Parallelism -- Chapter 7: Implementing Concurrency -- Technical requirements -- Asynchronous programming -- Waiting for input/output -- Concurrency -- Callbacks -- Futures -- Event loops -- The asyncio framework -- Coroutines -- Converting blocking code into non-blocking code -- Reactive programming -- Observables -- Useful operators -- Hot and cold observables -- Building a CPU monitor -- Summary -- Questions -- Further reading -- Chapter 8: Parallel Processing -- Technical requirements -- Introduction to parallel programming -- GPUs -- Using multiple processes
- Section 3: Design Patterns in Python -- Chapter 16: The Factory Pattern -- Technical requirements -- Understanding design patterns -- Implementing the factory method -- Real-world examples -- Use cases -- Implementing the factory method -- Applying the abstract factory -- Real-world examples -- Use cases -- Implementing the abstract factory pattern -- Summary -- Questions -- Chapter 17: The Builder Pattern -- Technical requirements -- Understanding the builder pattern -- Real-world examples -- Use cases -- Implementing an ordering application -- Summary -- Questions -- Chapter 18: Other Creational Patterns -- Technical requirements -- Implementing the prototype pattern -- Real-world examples -- Use cases -- Implementation -- Implementing the singleton pattern -- Real-world examples -- Use cases -- Implementation -- Summary -- Questions -- Further reading -- Chapter 19: The Adapter Pattern -- Technical requirements -- Understanding the adapter pattern -- Real-world examples -- Use cases -- Implementation -- Summary -- Chapter 20: The Decorator Pattern -- Technical requirements -- Introducing the decorator pattern -- Real-world examples -- Use cases -- Implementation -- Summary -- Questions -- Chapter 21: The Bridge Pattern -- Technical requirements -- Real-world examples -- Use cases -- Implementation -- Summary -- Questions -- Chapter 22: The Façade Pattern -- Technical requirements -- Understanding the façade pattern -- Real-world examples -- Use cases -- Implementation -- Summary -- Questions -- Further reading -- Chapter 23: Other Structural Patterns -- Technical requirements -- Implementing the flyweight pattern -- Real-world examples -- Use cases -- Implementation -- Implementing the model-view-controller pattern -- Real-world examples -- Use cases -- Implementation -- Applying the proxy pattern -- Real-world examples -- Use cases
- The Process and Pool classes -- The Executor interface -- Monte Carlo approximation of pi -- Synchronization and locks -- Parallel Cython with OpenMP -- Automatic parallelism -- Getting started with Theano -- Profiling Theano -- TensorFlow -- Running code on a GPU -- Summary -- Questions -- Chapter 9: Concurrent Web Requests -- The basics of web requests -- HTML -- HTTP requests -- HTTP status code -- The requests module -- Making a request in Python -- Running a ping test -- Concurrent web requests -- Spawning multiple threads -- Refactoring request logic -- The problem with timeouts -- Support from httpstat.us and simulation in Python -- Timeout specifications -- Good practices in making web requests -- Consider the terms of service and data-collecting policies -- Error handling -- Update your program regularly -- Avoid making a large number of requests -- Summary -- Questions -- Further reading -- Chapter 10: Concurrent Image Processing -- Technical requirements -- Image processing fundamentals -- Python as an image processing tool -- Computer image basics -- OpenCV API -- Image processing techniques -- Applying concurrency to image processing -- Good concurrent image processing practices -- Choosing the correct way (out of many) -- Spawning an appropriate number of processes -- Processing input/output concurrently -- Summary -- Questions -- Further reading -- Chapter 11: Building Communication Channels with asyncio -- Technical requirements -- The ecosystem of communication channels -- Communication protocol layers -- Asynchronous programming for communication channels -- Transports and protocols in asyncio -- The big picture of asyncio's server client -- Getting started with Python and Telnet -- Starting a server -- Installing Telnet -- Simulating a connection channel -- Sending messages back to clients -- Closing transports
- Client-side communication with aiohttp -- Installing aiohttp and aiofiles -- Fetching a website's HTML code -- Writing files asynchronously -- Summary -- Questions -- Further reading -- Chapter 12: Deadlocks -- Technical requirements -- The concept of deadlocks -- The dining philosophers problem -- A deadlock in a concurrent system -- Python simulation -- Approaches to deadlock situations -- Implementing ranking among resources -- Ignoring locks and sharing resources -- The concept of livelocks -- Summary -- Questions -- Further reading -- Chapter 13: Starvation -- Technical requirements -- Understanding starvation -- What is starvation? -- Scheduling -- Causes of starvation -- Starvation's relationship to deadlock -- Approaching the readers-writers problem -- Problem statement -- The first readers-writers problem -- The second readers-writers problem -- The third readers-writers problem -- Solutions to starvation -- Summary -- Questions -- Further reading -- Chapter 14: Race Conditions -- Technical requirements -- The concept of race conditions -- Critical sections -- How race conditions occur -- Simulating race conditions in Python -- Locks as a solution to race conditions -- The effectiveness of locks -- Implementation in Python -- The downside of locks -- Race conditions in real life -- Security -- Operating systems -- Networking -- Summary -- Questions -- Further reading -- Chapter 15: The Global Interpreter Lock -- Technical requirements -- Introducing the GIL -- Analyzing memory management in Python -- The problem that the GIL addresses -- Problems raised by the GIL -- The potential removal of the GIL from Python -- Working with the GIL -- Implementing multiprocessing, rather than multithreading -- Getting around the GIL with native extensions -- Utilizing a different Python interpreter -- Summary -- Questions -- Further reading
- Implementation
- Advanced Python Programming: Accelerate your Python programs using proven techniques and design patterns