CS231n笔记 Lecture 8, Deep Learning Software

Posted ichn

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了CS231n笔记 Lecture 8, Deep Learning Software相关的知识,希望对你有一定的参考价值。

CPU and GPU

技术分享图片

If you aren’t careful, training can bottleneck on reading data and transferring to GPU! Solutions:

  • - Read all data into RAM
  • - Use SSD instead of HDD
  • - Use multiple CPU threads to prefetch data

The point of deep learning frameworks

  • Easily build big computational graphs
  • Easily compute gradients in computational graphs
  • Run it all efficiently on GPU (wrap cuDNN, cuBLAS, etc)

 

DL frameworks

Pytorch大法好

TensorFlow

First define the graph, and then run it many times.

TOO UGLY!!! Introduces a lot of terms that doesn‘t seem to be important if it is designed right. And the api is not pythonic at all!

技术分享图片

use tensorboard to make life easier!

 

PyTorch

Pytorch大法好 

Tensor: ndarray that can do computations on GPU

Variable: node in a computational graph that supports Autograd. 

  • x.data. Tensor
  • x.grad. Variable of gradients with the same size of x.data
  • x.grad.data. the Tensor of gradients

nice and clean!

torch.nn package

  • already defined layers
  • build model on layers

torch.optim

update automatically with various optimization algorithms.

torchvision

pretrained models

Visdom

visualization.

 

技术分享图片

 

以上是关于CS231n笔记 Lecture 8, Deep Learning Software的主要内容,如果未能解决你的问题,请参考以下文章

cs231n spring 2017 lecture9 听课笔记

CS231n笔记 Lecture 9, CNN Architectures

CS231n笔记 Lecture 11, Detection and Segmentation

cs231n spring 2017 lecture11 听课笔记

cs231n spring 2017 lecture13 听课笔记

cs231n spring 2017 lecture3 听课笔记