Research on the efficiency of parallelization of processor-dependent tasks in Python based on the thread mechanism

Abstract

The objective of the paper is to study the effectiveness of using parallel data processing based on threads in the Python language to solve problems that require significant CPU resources. The task of processing a two-dimensional array of large sizes using standard Python tools without using specialized libraries such as Numpy was considered as such a problem. As a result of the research, an unexpected decrease in the execution time of the multi-threaded program by 30% was found in comparison with the sequential (non-parallel) implementation of the problem being solved. This makes it possible to question the well-known statement that multi-threaded solving of processor-dependent tasks in the Python language is not effective, as it does not lead to a decrease in the running time of programs. Previously, it was believed that due to the use of the so-called global lock (GIL), the Python language interpreter, when executing multithreaded programs using Threading module, directs all program threads to one processor core, which excludes the physical parallel execution of threads and under no circumstances allows to reduce time of program execution. A well-known recommendation for obtaining a real gain in time is the use of parallelization based on the process mechanism and the "multiprocessing" module, which allows several physical processor cores to be used. However, this approach requires additional processor resources (cores) and additional time spent on data exchange between processes, which can nullify the effect of multi-core processing. The experiments conducted by the authors proved that the use of a multithreaded approach can also be appropriate in the case of processor-dependent tasks, since the time gain achieved does not require additional resources from the computer's central processor and unnecessary time spent on data exchange between threads.