University:
Boston University
Course:
GRS CS 910 | Directed Study: Systems
Academic year:

2022
Views:

263

Pages:

15
Author:

Marely Dougherty

Using SciPy  Think about your code and what sort of algorithms you’re using:  Integration, linear algebra, image processing, etc.  See if an appropriate algorithm exists in SciPy before trying to write your own.  Read the docs – many functions have large numbers of optional arguments.  Understand the algorithms! Example: Fit a line with SciPy  There are many ways to fit equation parameters to data in NumPy and SciPy  scipy.stats.linregress: Calculate a regression line  Open the example linregress.py  This demonstrates calling the function and extracting all the info it returns. Example: scipy.optimize.minimize  Finds the minimum value of a function.  You provide the function as a variable to minimize.  This is a common pattern in scipy.  Open scipy_minimize.py 𝑦 = 3𝑥 2 + 𝑥 − 1 OpenCV  The Open Source Computer Vision Library  Highly optimized and mature C++ library usable from C++, Java, and Python.  Cross platform: Windows, Linux, Mac OSX, iOS, Android • • • • • • • • • • • • • Image Processing Image file reading and writing Video I/O High-level GUI Video Analysis Camera Calibration and 3D Reconstruction 2D Features Framework Object Detection Deep Neural Network module Machine Learning Clustering and Search in Multi-Dimensional Spaces Computational Photography Image stitching OpenCV vs SciPy  For imaging-related operations and many linear algebra functions there is a lot of overlap between these two libraries.  OpenCV is frequently faster, sometimes significantly so.  The OpenCV Python API uses NumPy ndarrays, making OpenCV algorithms compatible with SciPy and other libraries. OpenCV vs SciPy  A simple benchmark: Gaussian and median filtering a 1024x671 pixel image of the CAS building.  Gaussian: radius 5, median: radius 9.  Timing: 2.4 GHz Xeon E5-2680 (Sandybridge) Operation Function scipy.ndimage.gaussian_filter Time (msec) OpenCV speedup 85.7 Gaussian 3.7x cv2.GaussianBlur 23.2 scipy.ndimage.median_filter 1,780 Median 22.5x cv2.medianBlur 79.2 When NumPy and SciPy aren’t fast enough  Auto-compile your Python code with the numba and numexpr libraries  Use the Intel Python distribution  Re-code critical paths with Cython  Combine your own C++ (with SWIG) or Fortran code (with f2py) and call from Python numba  The numba library can translate portions of your Python code and compile it into machine code on demand.  Achieves a significant speedup compared with regular Python.  Compatible with numpy ndarrays.  Can generate code to execute automatically on GPUs. numba from numba import jit  The @jit decorator is used to indicate which functions are compiled.  Options: # This will get compiled when it's first executed @jit def average(x, y, z): return (x + y + z) / 3.0  GPU code generation  Parallelization  Caching of compiled code  Can produce faster array code than pure NumPy statements. # With type information this one gets # compiled when the file is read. @jit (float64(float64,float64,float64)) def average_eager(x, y, z): return (x + y + z) / 3.0 numexpr  Another acceleration library for Python.  Useful for speeding up specific ndarray expressions.  Typically 2-4x faster than plain NumPy  Code needs to be edited to move ndarray expressions into the numexpr.evaluate function: import numpy as np import numexpr as ne a = np.arange(10) b = np.arange(0, 20, 2) # Plain NumPy c = 2 * a + 3 * b # Numexpr d = ne.evaluate("2*a+3*b") Intel Python  Intel now releases a customized build of Python 2.7 and 3.6 based on their optimized libraries.  Can be installed stand-alone or inside of Anaconda: https://software.intel.com/en-us/distribution-for-python  Available on the SCC: module avail python2-intel (or python3-intel) Intel Python  In RCS testing on various projects the Intel Python build is always at least as fast as the regular Python and Anaconda modules on the SCC.  In one case involving processing several GB’s of XML code it was 20x faster!  Easy to try: change environments in Anaconda or load the SCC module.  Can use the Intel Thread Building Blocks library to improve multithreaded Python programs: python -m tbb parallel_script.py Cython  Cython is a superset of the Python language.  The additional syntax allows for C code to be auto-generated and compiled from Python code.  This can make mixing Python, Cython, and C code (or libraries) very straightforward.  A mature library that is widely used. You feel the need for speed…  Auto-compilation systems like numba, numexpr, and Cython:  all provide access to higher speed code  minimal to significant code changes  You’re still working in Python or Python-like code  Faster than NumPy which is also much faster than plain Python for numeric calculation  For the fastest implementation of algorithms, optimized and well-written C, C++, and Fortran codes are very hard to beat  Connect C or C++ to Python with SWIG  Connect Fortran to Python with f2py (part of Numpy)  Contact RCS for help! End-of-course Evaluation Form  Please visit this page and fill in the evaluation form for this course.  Your feedback is highly valuable to the RCS team for the improvement and development of tutorials.  If you visit this link later please make sure to select the correct tutorial – name, time, and location.

Numerical and Scientific Computing in Python Part 3

Related Documents

Recommended Documents

Description

Related Documents

Free up your schedule!

Take 5 seconds to unlock