Skip to main content

Posts

Precision Recall and PR-AUC

 https://towardsdatascience.com/gaining-an-intuitive-understanding-of-precision-and-recall-3b9df37804a7 Precision = True Positives / All Predicted Positives Recall = True Positives / All Actual Positives Precision = TP / (TP + FP) Precision doesn't care about False Negatives (We are only looking at positive class now so we ignore negative correct prediction.) Meaning if a lot of positive class is classified as negative, precision can still be high. So by using high threshold precision can be made 1. So precision alone is not a good measure. Recall =TP / (TP + FN) Recall is like accuracy of positive class only. So by making all classes predict positive (1), recall can be made very high. For example, if threshold is 0 and all classes are predicted as positive (1), recall is high. So recall alone is also not a good measure of classifier performance. So if we move towards 0 threshold, recall increases, precision decreases., typically. If we move towards 1 threshold, precision increases...

Combine Dictionaries

 #combine dict dict1 = {'BH': ['apple1', 'banana1'], 'FB': ['orange1', 'grapes1']} dict2 = {'BH': ['apple2', 'banana2'], 'FB': ['orange2', 'grapes2']} dict3 = {} for k in dict1.keys():     dict3[k] = ['one/' + itm for itm in dict1[k]] + ['two/' + itm for itm in dict1[k]] print(dict3)

Airpods mic not working with bluetooth.

 It's because in playback, Headphones was selected as default device and not Headset Headphones: No Mic Headset: With Mic So go to sound: Playback: Set Headset as default communications device and default playback device both Sometimes the mic is very terrible, in that case plug out bluetooth adapter and plug in again.

Placeholder Shape - Tensorflow

 Placeholder can be defined as: tf.placeholder(tf.float32, shape=(1, 16, 64, 64, 1), name='real_node_a')  or tf.placeholder(tf.float32, shape=(None, None, None, None, None), name='real_node_a')  If you want to read shape from None, use: h      = tf.shape(inputs)[2] w      = tf.shape(inputs)[3] If you want to read shape from numbers, use: nb, nt = inputs.get_shape().as_list()[0:2] sometimes some nodes in the graph require integer input instead of None, in that case you must hard code placeholder with numbers and use get_shape None provides more flexibility but might not work with all.

Makefile / Make

 https://sourceforge.net/projects/gnuwin32/ Install this and add C:\Program Files (x86)\GnuWin32\bin to path Then you can run make from command prompt / powershell https://makefiletutorial.com/ For make, You will need a 'Makefile' in the folder where you are trying to run make from. make is basically like an automation script

tf.nn vs tf.layers

 https://stackoverflow.com/questions/42785026/tf-nn-conv2d-vs-tf-layers-conv2d https://stackoverflow.com/questions/33727935/how-to-use-stop-gradient-in-tensorflow https://www.cs.toronto.edu/~frossard/vgg16/vgg16.py

VS Code

 After opening output window Select default profile to command prompt add anaconda to path env varibles Example: C:\Anaconda3\condabin

Open ipynb on double click

 https://axil.github.io/how-to-open-ipynb-file-with-one-doubleclick-on-windows.html Can install this inside conda env: pip install nbopen python -m nbopen.install_win This will reuse existing jupyter server if possible (=if it is already launched in the same dir). Then two options: (I did the second) Run this command in cmd.exe under administrator privileges: assoc .whl=jupyter& ftype jupyter=cmd.exe /c jupyter-notebook "%1" Alternatively, a slightly different line can be copied into a assoc_ipynb.bat file and executed through 'Run as administrator': assoc .whl=jupyter& ftype jupyter=cmd /c jupyter-notebook "%%1" PS jupyter-notebook.exe is assumed to be in the PATH. to close open notebooks open conda prompt any env jupyter notebook list jupyter notebook stop 8888

3D Animation Matlab

 import matplotlib.pyplot as plt import matplotlib.animation as animation %matplotlib inline fig = plt.figure(dpi=200) ims = [] for t in range(0, img1.shape[0]):     imx = img1[t,:,:]     im = plt.imshow(imx, cmap='gray', animated=True)     ims.append([im]) ani = animation.ArtistAnimation(fig, ims, interval=200, blit=True,                                 repeat_delay=1000) plt.close() from IPython.display import HTML HTML(ani.to_jshtml()) # if you want to keep inside function you can use display(HTML(ani.to_jshtml())) Example: def playVideo(input_sample):     numFrames = input_sample.shape[0]     fig = plt.figure()     fig.set_size_inches(10, 10, True)     im = plt.imshow(input_sample[0,:,:])     plt.close()     def update(i):         img = input_sample[i,:,:]         im...

Tensorflow and Pytorch for Cuda 10.0

And about the cuda, I believe you are setting up for Cuda 10.0 right? If that's the case I think the steps would be to install VS 2017, then CUDA and then download CUDNN (of compatible version) and copy it to the CUDA folder. also you might need to add cuda dir to system paths after copying cudnn. For, VS 2017 - tensorflow_gpu-1.15.0 cuDNN=7.4 CUDA=10 VS 2017: https://download.visualstudio.microsoft.com/download/pr/010d871e-3fa8-4004-b219-1c10bcaa71ad/a3ee8ecda3f9cc003dfe40db87eb56d38b21087116f51e243ab0455348f4b188/vs_Community.exe CUDA: https://developer.nvidia.com/compute/cuda/10.0/Prod/network_installers/cuda_10.0.130_win10_network CuDNN: https://developer.nvidia.com/cudnn Download cuDNN v7.6.5 (November 5th, 2019), for CUDA 10.0 Then: conda create -n iq42mas python=3.6.5 conda install -c anaconda cudatoolkit=10.0 pip install tensorflow-gpu==1.15 conda install pytorch==1.2.0 torchvision==0.4.0 cudatoolkit=10.0 -c pytorch

Tensorflow GPU Check

import tensorflow as tf with tf.device('/gpu:0'):       a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')       b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')       c = tf.matmul(a, b) with tf.Session() as sess: print(sess.run(c))

Make decompiling python code, generated from pyinstaller harder.

 General idea: Pyinstaller when creates an executable generates .pyc files of the functions or scripts. These .pyc files are easy to decompile. So a better security option is to use .pyd files. We need cython to generate .pyd files. Read: https://medium.com/analytics-vidhya/how-to-create-executable-of-your-python-application-from-linux-windows-mac-bcbcdd4603d4 https://programming.vip/docs/decompile-pyinstaller-packaged-exe-installation-package.html https://github.com/extremecoders-re/pyinstxtractor/ Step 1: creating an executable .exe file. Need: pyinstaller pip install pyinstaller then run: pyinstaller yourfile.py this will create an exe field in the dist directory - dist is the dir we are interested in not build but in the dist dir there are supporting files as well, to create a single executable file use: pyinstaller yourfile.py --onefile or  pyinstaller -F simpleprogram.py The problem: This exe can be decompiled to view source code. Decompiling: https://github.com/extremec...

Python Class Call

 https://www.geeksforgeeks.org/__call__-in-python/ __call__ in Python object() is shorthand for object.__call__() class Example:      def __init__(self):          print("Instance Created")             # Defining __call__ method      def __call__(self):          print("Instance is called via special method")     # Instance created  e = Example()     # __call__ method will be called  e()  Output : Instance Created Instance is called via special method class Product:      def __init__(self):          print("Instance Created")         # Defining __call__ method      def __call__(self, a, b):          print(a * b)     # Instance created  ans = Product()     # __call__ method will be called...

Generators Python

https://www.programiz.com/python-programming/generator https://www.programiz.com/python-programming/methods/built-in/next Python Generators When a lot of data is present and you cannot load it into the memory at once Or if a function needs to maintain an internal state everytime it's called yield: yield is used in conjunction with generators Generators are functions They return lazy iterators.  lazy evaluation, or call-by-need, is an evaluation strategy which delays the evaluation of an expression until its value is needed (non-strict evaluation) and which also avoids repeated evaluations (sharing).  The sharing can reduce the running time of certain functions by an exponential factor over other non-strict evaluation strategies, such as call-by-name, which repeatedly evaluate the same function, blindly, regardless whether the function can be memoized. Iterators are objects that you can loop over like a list But unlike lists, lazy iterators do not store their contents...

Staticmethod python

 Defined as @staticmethod def func(args, ...) @staticmethod  = decorator static methods do not have access to the properties of the class itself but you want them to belong to the class For example a utility function which might be useful for a class but you don't want it to interfere with other things in the class

Matplotlib Subplots

# fig = plt.figure() fig = plt.figure(figsize=(50,50)) ax = fig.add_subplot(1, 2, 1) plt.imshow(imgArray1, cmap='gray', vmin = 0, vmax = 255, interpolation='none', aspect='auto') #remove cmap to see colors ax.set_title('title1') ax = fig.add_subplot(1, 2, 2) plt.imshow(imgArray2, cmap='gray', vmin = 0, vmax = 255, interpolation='none') plt.show()  # # Uncomment to save image and see scores on title of image # picoutpath = os.path.join(self.outdir, 'lax_classify_jpg_out', labeldir) # os.makedirs(picoutpath, exist_ok=True) # picoutpath = os.path.join(picoutpath, filename.replace('.npy', '.jpg')) # plt.imshow(imgarr, cmap='gray', interpolation=None) # plt.title(f' [ GT:{gtrth_x} PD:{predc_x} PRB:{ma_score:.2f} ]') # plt.savefig(picoutpath, dpi=200) # plt.close() fig, axs = plt.subplots(1, 2, dpi=200)     i = 0      im = axs[i].imshow(img, cmap='gray', interpolation=None)     axs[i].set_title(f...

Anaconda Prompt and Sublime Text - Windows (Add to right click)

Source:  https://gist.github.com/jiewpeng/8ba446acf329b1801bf91db767d179ea Open Regedit. Win + R > regedit.exe > Enter. Go to: Computer\HKEY_CLASSES_ROOT\Directory\Background\shell\ Add a key named AnacondaPrompt and set its value (Default) to "Anaconda Prompt Here" (or anything you'd like it to appear as in the right click context menu) Add a key under this key, called command, and set its value to: cmd.exe /K C:\Users\rb\Anaconda3\Scripts\activate.bat (may have to change the activate.bat file to whereever Anaconda is installed) No need to restart. To setup conda env to another dir: https://stackoverflow.com/questions/67610133 conda config --show conda config --add pkgs_dirs D:/.conda/pkgs conda config --add envs_dirs D:/.conda/envs conda config --remove envs_dirs  C:/Users/rb/.conda/envs conda config --remove envs_dirs  C:\Users\rb\AppData\Local\conda\conda\envs Sublime : Create a new file called foo.reg, paste the following text & execut...

Caffe : Softmax with Loss

Let us assume 2x2 image, 3 channels (labels possibilities) and 2 batch data. Then the predicted scores (logits) is given by: Assume 2x2 array is arranged in a single row. For N = 1 (of batch 2): [       9       2       5       8       ]       Each row is a class [       7       8       4       6       ]       Each column is score of scores of every class for a pixel [       8       4       6       7       ] For N = 2 (of batch 2): [       7       3       0       3       ] [       4       6    ...

Uninstalling CUDA 10.0 cuDNN 7.4.2 on Ubuntu 18.04 and Install CUDA 8.0 cuDNN 5.1

https://davidstutz.de/upgrading-cuda-and-installing-cudnn-for-caffe-and-tensorflow/ Check Ubuntu Version: lsb_release -a Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic If NVIDIA Driver is not installed: cd ~/Downloads sudo sh ./NVIDIA-Linux-x86_64-410.104.run register the kernel module sources with dkms - no 32 bit - no X config - yes Check CUDA version: nvcc --version Check cuDNN version: whereis cudnn.h cudnn: /usr/include/cudnn.h cat /usr/include/cudnn.h | grep CUDNN_MAJOR -A 2 #define CUDNN_MAJOR 7 #define CUDNN_MINOR 4 #define CUDNN_PATCHLEVEL 2 -- #define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL) #include "driver_types.h" which means the version is 7.4.2. Uninstalling: Find the CUDA dir using: which nvcc Mine was: rb@rbhost:~$ which nvcc /usr/local/cuda-10.0/bin/nvcc To uninstall the CUDA Toolkit, run the uninstall script in `/usr/local/cuda-10.0/bin`...

Install Caffe with GPU

The instruction are for: Caffe3D : Ubuntu, Caffe (2D) : Windows. The Caffe3D author did it in python 2.7, Ubuntu 14.04, CUDNN 5.1, CUDA 8.0 Read the errors at the end before you continue. And always make the Makefile.config yourself don't just copy paste. check for the folders carefully. Install Caffe 3D: Download  https://lmb.informatik.uni-freiburg.de/resources/opensource/caffe_unet_3D_v1.0.tar.gz Extract. Open README: git clone https://github.com/BVLC/caffe.git cd caffe git checkout 8c66fa5f3c04e -b unet_patch git cherry-pick 458928a # typo in installation.md git cherry-pick b43c8e4  # CuDNN 5 support git apply ../caffe_unet_3D_v1.0.patch # apply patch git add . git commit -m"U-Net 3D merged to BVLC/caffe 8c66fa5f3c04e" Then open: https://github.com/tbuikr/3D_DenseSeg Edit different files as mentioned there. Now need to compile for linux now: https://github.com/adeelz92/Install-Caffe-on-Ubuntu-16.04-Python-3 Error: libcudnn.so.5 is not a symbolic...