Algidus

TORCH torch.backends.cudnn.benchmark=True will enable the inbuilt cudnn auto-tuner to find the best algorithm to use for your hardware. This way, cudnn will look for the optimal set of algorithms for that particular configuration (which takes some time). This usually leads to faster runtime. It depends on the task. If your input size is changing a lot, then it might hurt runtime, if not, it should be much faster. https://discuss.pytorch.org/t/what-does-torch-backends-cudnn-benchmark-do/5936/3 [https://hsaghir.github.io/data_science/pytorch_starter/] torch: a general purpose array library similar to Numpy that can do computations on GPU when the tensor type is cast to (torch.cuda.TensorFloat) torch.autograd: a package for building a computational graph and automatically obtaining gradients torch.nn: a neural net library with common layers and cost functions torch.optim: an optimization package with common optimization algorithms like SGD,Adam, etc torch.j...

Python Class: Class has attributes. self.fname, self.lname are called attributes. Class has methods. Functions inside class are called methods. Say we have a Person class with attributes fname and lname and method getname which gives fname + lname Inheritance: Say we want to create a class Employee, which needs attributes fname, lname and id and methods getname and getid We can create it from scratch with these attributes and methods. Or we can inherit Person which already has fname, lname and getname. Prev: class Person: define __init__(self, fname, lname) and self.fname = fname and self.lname = lname. Now:  class Employee(Person): define __init__ (self, fname, lname, id) and Person.__init__(self,fname,lname) then you can call self.getname, self.fname, which is from Person class. Person.__init__(self,fname,lname)  #Avoid this Limits your ability to use multiple inheritance The __init__ method of our Employee class explicitly invokes the __init__method of t...

sudo apt install conky-all https://help.ubuntu.com/community/SettingUpConky https://linuxconfig.org/system-monitoring-on-ubuntu-18-04-linux-with-conky enable conky to automaticaly start at the boot: Open Startup Applications Preferences from Activities. Click Add Name: Conky System Monitoring Command: /usr/bin/conky Click Add Reboot or re-login into your Ubuntu 18.04 system. Go to: /etc/conky Open: conky.conf as admin Simply, sudo nano /etc/conky/conky.conf Change alignmentt = 'top_right' Ctrl + O to save and Ctrl + X to exit or another way to change it is: sudo sed -i 's/left/right/' /etc/conky/conky.conf This will change conky.conf for all users in ubuntu. To make it specific for a user only do: cp /etc/conky/conky.conf ~/.conkyrc The above command created a user based Conky configuration file located at ~/.conkyrc. To apply changes, re-login or reboot. To make network monitor work, need to change eth0 with your network interface name: rb...

Source: https://medium.com/@UdacityINDIA/tensorflow-or-pytorch-the-force-is-strong-with-which-one-68226bb7dab4 1: Tensorflow based on Theano developed by Google PyTorch based on Torch developed by Facebook 2. Tensorflow is static - design a computational graph first then run ML model. PyTorch is dynamic - you can define/manipulate your graph on-the-go. Helpful while using variable length inputs in RNNs. 3. PyTorch is a relatively new framework as compared to Tensorflow - so small community. TensorBoard -enables visualizing your ML models directly in your browser. PyTorch doesn’t have such a tool, although you can always use tools like Matplotlib PyTorch is easier to learn and lighter to work with, and hence, is relatively better for passion projects and building rapid prototypes.

If you want to close other sessions used by GPU: if 'session' in locals() and session is not None:      print('Close interactive session')      session.close() Reinstall nvidia driver if you don't see nvidia-smi working. If you see: failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED or InternalError: Blas GEMM launch failed : a.shape= Try: sudo rm -rf .nv/ This probably clears the nvidia cache https://github.com/tensorflow/tensorflow/issues/5354 if you're still having trouble, try adding /usr/local/cuda/extras/CUPTI/lib64 to your LD_LIBRARY_PATH. I had the same error and this fixed it (I was on mac though, so verify that directory on your system) export tfhome="/media/rb/Omega/tensorflow" export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}} export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} export LD_LIBRARY_PATH=/usr/local/cuda-10.0/extras/CUPTI/lib64${LD_LIBRARY_PATH:+:${LD_LI...

https://medium.com/usf-msds/deep-learning-best-practices-1-weight-initialization-14e5c0295b94

pip install <package> --upgrade --ignore-installed pip install --upgrade --force-reinstall --ignore-installed <package> if you want to downgrade to a version conda install jupyter notebook pip install notebook --upgrade [because default notebook in conda is installed by pip] conda upgrade notebook pip install matplotlib pip install scikit-image  [for skimage] pip install scikit-learn [for sklearn] pip install imageio pip install numpy opencv-python lmdb pip install tensorboardX pip install utils pip install torchsummary pip install h5py pip install torchvision pip install pandas pip install path.py pip install imgaug                      # https://github.com/aleju/imgaug pip install cython git clone https://github.com/waleedka/coco.git cd coco/PythonAPI pip install pycocotools Prerequisites for imgaug [in linux didn't need]: pip install six numpy scipy Pillow matplotlib scikit-image opencv-...

Source: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/protobuf/config.proto https://stackoverflow.com/questions/44873273/ https://www.tensorflow.org/guide/using_gpu 1. allow_soft_placement=True Some tensorflow operations have GPU implementations, like MatMul, some don't. // Whether soft placement is allowed. If allow_soft_placement is true, // an op will be placed on CPU if //   1. there's no GPU implementation for the OP // or //   2. no GPU devices are known or registered // or //   3. need to co-locate with reftype input(s) which are from CPU. bool allow_soft_placement = 7; So, if you do : with tf.device('/gpu:0'): // And you have some op here which doesn't have a GPU implementation This will throw an error if allow_soft_placement=True is not set. 2. log_device_placement=True // Whether device placements should be logged. bool log_device_placement = 8; If log_device_placement=True You can see where e...

I have only one GPU so all discussions are single GPU only, no multi GPU Tensorflow uses GPU by default , when one is available. Example: a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a') b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b') c = tf.matmul(a, b) sess = tf.Session(config=tf.ConfigProto(log_device_placement=True)) print(sess.run(c)) # log_device_placement True will show which devices does which operation Here, MatMul: (MatMul)/job:localhost/replica:0/task:0/device:GPU:0 a: (Const)/job:localhost/replica:0/task:0/device:GPU:0 b: (Const)/job:localhost/replica:0/task:0/device:GPU:0 matmul has both CPU and GPU kernels, by default GPU is selected Running on CPU: with tf.device('/cpu:0'):      a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')      b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')      c = tf.matmul(a, b) sess...

Some cool commands: nvidia-smi, neofetch, watch -n1 nvidia-smi, anaconda-navigator, conda info --envs, conda remove -n yourenvname --all C:\Program Files\NVIDIA Corporation\NVSMI watch -d -n 0.5 nvidia-smi Note: Install TF and PyTorch using Update 2 only, rest is what I tried and failed. On Windows: Install Anaconda. Open Anaconda Prompt conda create --name tf_gpu tensorflow-gpu Source: https://towardsdatascience.com/tensorflow-gpu-installation-made-easy-use-conda-instead-of-pip-52e5249374bc python -c "import tensorflow as tf; print(tf.__version__)" Install Keras after TF: pip install keras    [Source https://www.quantinsti.com/blog/install-tensorflow-gpu] python -c "import keras; print(keras.__version__)" conda create --name torch activate torch conda install pytorch -c pytorch Source: https://medium.com/@bryant.kou/how-to-install-pytorch-on-windows-step-by-step-cc4d004adb2a Linux: python Python 3.7.4 (default, Aug 13 2019, 20:35:4...

Virtual Environment Python Installing Tensorflow GPU https://algidus.blogspot.com/p/installing-tensorflow-gpu-2.html