写在最前:最近印象比较深的一句话:
长期灌水垃圾sci论文会让你养成惰性,最终你将永远写不出高质量的论文 ------代彬
虽然我不太可能写出高质量的sci,但是关于自己的学习笔记应该要写的很认真。
以前认识不到做笔记的重要性,认为只要听懂了就好了。
笔记是为了自己整理思路,如果囫囵吞枣的学一遍,没有一个知识图谱,很快将会忘掉。
这篇多进程和多线程在网上一搜可能能搜到各种资源,讲的比我清楚的多,上一句就是我要做笔记的原因。
以上知识如果我在高中的时候知道,也许会去个更好的学校吧。
我之前写的东西,质量都比较差。
具体体现在:注释不够清楚,解释不够详细。纯粹为了凑数量,外行看来确实写了很多。
写笔记这件事还是不能骗自己。
不bibi了
多线程: 通俗解释:计算机处理一大堆数据。计算机太懒了,觉得一个人处理这么多要花费很多时间。于是找了很多人都一块来处理。那么多人来处理这整个数据就叫多线程。我们可以简单理解为一条生产线上的所有工人。
特点Python简单代码实现
<p><pre> <code class="language-python"><span class="kn">import</span> <span class="nn">threading</span> <span class="c1"># python多进程的包 </span>
<span class="k">def</span> <span class="nf">thread_job</span><span class="p">(</span><span class="n">a</span><span class="p">):</span>
<span class="k">global</span> <span class="n">lock</span> <span class="c1"># 将线程锁引入</span>
<span class="n">lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span> <span class="c1"># 开启线程锁</span>
<span class="k">print</span><span class="p">(</span><span class="s1">'This is an added Thread, number is </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">threading</span><span class="o">.</span><span class="n">current_thread</span><span class="p">())</span> <span class="c1"># 打印当前线程</span>
<span class="k">print</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="n">lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span> <span class="c1"># 释放线程锁</span>
<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
<span class="c1"># 激活了多少线程</span>
<span class="k">print</span><span class="p">(</span><span class="n">threading</span><span class="o">.</span><span class="n">active_count</span><span class="p">())</span>
<span class="c1"># 是哪些线程</span>
<span class="k">print</span><span class="p">(</span><span class="n">threading</span><span class="o">.</span><span class="n">enumerate</span><span class="p">())</span>
<span class="c1"># 当前线程</span>
<span class="k">print</span><span class="p">(</span><span class="n">threading</span><span class="o">.</span><span class="n">current_thread</span><span class="p">())</span>
<span class="c1"># 添加thread</span>
<span class="n">a</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">added_thread</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Thread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">thread_job</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">a</span><span class="p">,)</span> <span class="n">name</span><span class="o">=</span><span class="s1">'T1'</span><span class="p">)</span> <span class="c1"># args 为传入的参数,必须为tuple类型。name为名字,可以自己定义</span>
<span class="c1"># 执行thread</span>
<span class="n">added_thread</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="c1"># 将此线程加到主线程上。表现的结果为主线程会是最后一个结束,不加的话在执行added_thread的job时,如果时间过长,表现为主线程先结束。</span>
<span class="n">added_thread</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
<span class="n">lock</span> <span class="o">=</span> <span class="n">threading</span><span class="o">.</span><span class="n">Lock</span><span class="p">()</span> <span class="c1"># 线程锁。意义在于如果有两个任务,第二个线程需要第一个线程返回的数据。则把第一个锁住。确保第一个执行了第二个词再执行。</span>
<span class="n">main</span><span class="p">()</span></code></pre></p>
多进程: 通俗解释:多进程就是拥有不同的生产线的工厂。
特点Python简单代码实现
<p><pre> <code class="language-python"><span class="kn">import</span> <span class="nn">multiprocessing</span> <span class="kn">as</span> <span class="nn">mp</span>
<span class="k">def</span> <span class="nf">job</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">d</span><span class="p">):</span>
<span class="k">print</span><span class="p">(</span><span class="s1">'aaaaa'</span><span class="p">)</span>
<span class="c1"># 与多线程很相似</span>
<span class="n">p1</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">job</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">),</span> <span class="n">name</span><span class="o">=</span><span class="s1">'P1'</span><span class="p">)</span>
<span class="n">p1</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">p1</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
<span class="o">--------------------------------------------------------------------------</span>
<span class="c1"># 进程池</span>
<span class="k">def</span> <span class="nf">job</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span><span class="o">*</span><span class="n">x</span>
<span class="k">def</span> <span class="nf">multicore</span><span class="p">():</span>
<span class="n">pool</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Pool</span><span class="p">(</span><span class="n">processes</span><span class="o">=</span><span class="mi">4</span><span class="p">)</span> <span class="c1"># 分配几个核</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">job</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span> <span class="c1"># map 表示可迭代的参数</span>
<span class="k">print</span><span class="p">(</span><span class="n">res</span><span class="p">)</span>
<span class="n">res</span> <span class="o">=</span> <span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">job</span><span class="p">,</span> <span class="p">(</span><span class="mi">2</span><span class="p">,))</span> <span class="c1"># apply_async 表示只能是一个参数,不可迭代</span>
<span class="k">print</span><span class="p">(</span><span class="n">res</span><span class="o">.</span><span class="n">get</span><span class="p">())</span>
<span class="c1"># 强行改成迭代的方法</span>
<span class="n">multi_res</span> <span class="o">=</span> <span class="p">[</span><span class="n">pool</span><span class="o">.</span><span class="n">apply_async</span><span class="p">(</span><span class="n">job</span><span class="p">,</span> <span class="p">(</span><span class="n">i</span><span class="p">,))</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">)]</span>
<span class="k">print</span><span class="p">([</span><span class="n">re</span><span class="o">.</span><span class="n">get</span><span class="p">()</span> <span class="k">for</span> <span class="n">re</span> <span class="ow">in</span> <span class="n">multi_res</span><span class="p">])</span>
<span class="o">--------------------------------------------------------------------------</span>
<span class="c1"># 共享内存</span>
<span class="n">value</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Value</span><span class="p">(</span><span class="s1">'d'</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">array</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Array</span><span class="p">(</span><span class="s1">'i'</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">])</span> <span class="c1"># 不可是多维的</span>
<span class="c1"># 共享内存和进程锁</span>
<span class="k">def</span> <span class="nf">job</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">num</span><span class="p">,</span> <span class="n">l</span><span class="p">):</span>
<span class="n">l</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span> <span class="c1"># 加上进程锁</span>
<span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.1</span><span class="p">)</span>
<span class="n">v</span><span class="o">.</span><span class="n">value</span> <span class="o">+=</span> <span class="n">num</span>
<span class="k">print</span><span class="p">(</span><span class="n">v</span><span class="o">.</span><span class="n">value</span><span class="p">)</span>
<span class="n">l</span><span class="o">.</span><span class="n">release</span><span class="p">()</span> <span class="c1"># 释放线程锁</span>
<span class="c1"># 这个方法可以实现累加,不会出现断断续续的加</span>
<span class="k">def</span> <span class="nf">multicore</span><span class="p">():</span>
<span class="n">l</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Lock</span><span class="p">()</span>
<span class="n">v</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Value</span><span class="p">(</span><span class="s1">'i'</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="c1"># 共享了个i类型的数据0,i可以查表知道什么类型</span>
<span class="n">p1</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">job</span><span class="p">,</span><span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">v</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="n">l</span><span class="p">))</span>
<span class="n">p2</span> <span class="o">=</span> <span class="n">mp</span><span class="o">.</span><span class="n">Process</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">job</span><span class="p">,</span><span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">v</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="n">l</span><span class="p">))</span>
<span class="n">p1</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">p2</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">p1</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
<span class="n">p2</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
<span class="n">multicore</span><span class="p">()</span></code></pre></p>
asyncio python 协程框架
通俗解释:IO密集型任务,一个用来协调的工具。
那么为什么要协调工作?
因为IO代表INPUT,OUTPUT很多,但是此时又不需要CPU的运算。也就是后来的代码全部需要前一个任务结束之后才能执行,很浪费时间。
举个例子:做爬虫时,在你请求第一个图片获得数据的时候,它会切换请求第二个图片或其他图片,等第一个图片获得所有数据后再切换回来,从而实现多线程批量下载的功能,速度超快。
本质:单线程python代码简单实现
<p><pre> <code class="language-python"><span class="kn">import</span> <span class="nn">asyncio</span> <span class="c1">#引入协程框架</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="n">async</span> <span class="k">def</span> <span class="nf">job</span><span class="p">(</span><span class="n">t</span><span class="p">):</span> <span class="c1"># async代表这个方法可以异步</span>
<span class="k">print</span><span class="p">(</span><span class="s1">'Start job '</span><span class="p">,</span> <span class="n">t</span><span class="p">)</span>
<span class="n">await</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="c1"># await代表运行这个方法的时候可以切换到下一个程序</span>
<span class="k">print</span><span class="p">(</span><span class="s1">'Job '</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span> <span class="s1">' takes'</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span> <span class="s1">' s'</span><span class="p">)</span>
<span class="n">async</span> <span class="k">def</span> <span class="nf">main</span><span class="p">(</span><span class="n">loop</span><span class="p">):</span>
<span class="n">tasks</span> <span class="o">=</span> <span class="p">[</span><span class="n">loop</span><span class="o">.</span><span class="n">create_task</span><span class="p">(</span><span class="n">job</span><span class="p">(</span><span class="n">t</span><span class="p">))</span> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">3</span><span class="p">)]</span> <span class="c1"># loop代表创建了协程的任务</span>
<span class="n">await</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">wait</span><span class="p">(</span><span class="n">tasks</span><span class="p">)</span>
<span class="n">t1</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
<span class="n">loop</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">get_event_loop</span><span class="p">()</span>
<span class="n">loop</span><span class="o">.</span><span class="n">run_until_complete</span><span class="p">(</span><span class="n">main</span><span class="p">(</span><span class="n">loop</span><span class="p">))</span>
<span class="n">loop</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">print</span><span class="p">(</span><span class="s1">'Async'</span><span class="p">,</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">t1</span><span class="p">,</span> <span class="s1">' s'</span><span class="p">)</span></code></pre></p>
参考
[1] 进程与线程的一个简单解释
[2] 莫烦python |
|