Algidus

Let us assume 2x2 image, 3 channels (labels possibilities) and 2 batch data. Then the predicted scores (logits) is given by: Assume 2x2 array is arranged in a single row. For N = 1 (of batch 2): [       9       2       5       8       ]       Each row is a class [       7       8       4       6       ]       Each column is score of scores of every class for a pixel [       8       4       6       7       ] For N = 2 (of batch 2): [       7       3       0       3       ] [       4       6    ...

Faststone https://www.faststone.org/FSIVDownload.htm Source: https://answers.microsoft.com/en-us/windows/forum/all/how-do-i-open-two-pictures-side-by-side/94ea5844-7aae-4baa-a75a-52dd0dfad950 Just select 2 to 4 images and press P to open the compare screen.

https://davidstutz.de/upgrading-cuda-and-installing-cudnn-for-caffe-and-tensorflow/ Check Ubuntu Version: lsb_release -a Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic If NVIDIA Driver is not installed: cd ~/Downloads sudo sh ./NVIDIA-Linux-x86_64-410.104.run register the kernel module sources with dkms - no 32 bit - no X config - yes Check CUDA version: nvcc --version Check cuDNN version: whereis cudnn.h cudnn: /usr/include/cudnn.h cat /usr/include/cudnn.h | grep CUDNN_MAJOR -A 2 #define CUDNN_MAJOR 7 #define CUDNN_MINOR 4 #define CUDNN_PATCHLEVEL 2 -- #define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL) #include "driver_types.h" which means the version is 7.4.2. Uninstalling: Find the CUDA dir using: which nvcc Mine was: rb@rbhost:~$ which nvcc /usr/local/cuda-10.0/bin/nvcc To uninstall the CUDA Toolkit, run the uninstall script in `/usr/local/cuda-10.0/bin`...

The instruction are for: Caffe3D : Ubuntu, Caffe (2D) : Windows. The Caffe3D author did it in python 2.7, Ubuntu 14.04, CUDNN 5.1, CUDA 8.0 Read the errors at the end before you continue. And always make the Makefile.config yourself don't just copy paste. check for the folders carefully. Install Caffe 3D: Download  https://lmb.informatik.uni-freiburg.de/resources/opensource/caffe_unet_3D_v1.0.tar.gz Extract. Open README: git clone https://github.com/BVLC/caffe.git cd caffe git checkout 8c66fa5f3c04e -b unet_patch git cherry-pick 458928a # typo in installation.md git cherry-pick b43c8e4  # CuDNN 5 support git apply ../caffe_unet_3D_v1.0.patch # apply patch git add . git commit -m"U-Net 3D merged to BVLC/caffe 8c66fa5f3c04e" Then open: https://github.com/tbuikr/3D_DenseSeg Edit different files as mentioned there. Now need to compile for linux now: https://github.com/adeelz92/Install-Caffe-on-Ubuntu-16.04-Python-3 Error: libcudnn.so.5 is not a symbolic...

Incomplete - Need to study more Batch Normalization is important to train deep nns. VGG 16 and 19 were developed before BN. So they trained 11 layer first added few layers trained again and so on. Inception used auxillary losses to train, not necessary but to propagate loss into first layers. Residual Nets: 1 important property, if the weights of residual network is zero, then it behaves as identity transformation, so that network can choose what is not required. Easy for the model to learn to not to use the layers it doesn't need. L2 regularization - making the weights to zero. 2 Gradient flow in backward pass is easy, so deeper nets can be designed. DenseNet and FractalNet - Study! Recurrent Neural Network   - Variable Size Data   x -------> [ RNN ] --------> y   Everytime x is inputted, RNN's hidden state will be updated, here's the difference: the internal hidden state will be fed back to the model on next input and so on. So input -> updat...

Cross Entropy Vs. Squared Error | KL-Divergence Generative Adversarial Network Generative model is about finding the underlying distribution or model from which the samples of images/data are supposed to come from. Say we have a million images of cats, so we can assume, all the images come from one distribution, Pcat So a sample from Pcat is an image of cat. So we want: to model Pcat. So we model the data / find the model of the data. Model = Distribution. [yt: hQv8FNaJHEA] Explicit model: Learns the actual parameter of the distribution. X_1 = P (X) --> Finding P Implicit model: Don't bother to learn the distribution of the data, instead focus on the stochastic [random] procedure that directly generate the data - on some input. X_1 = F_W(Z), Z = Latent space/input, W = Learned weights GAN are implicit models. Overall View: minG maxD { x~Pdata(X) E[ logD(X) ] + z~Pz(Z) E[ log ( 1 - D(G(Z)) )] X is a real data, Z is the noise, and G(Z) is...

[Source: https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html] output_size = ( I + P_left + P_right - D*(F-1) -1) / S + 1 I = Input Size, F = Filter Size, D = Dilation, S = Stride, P = Padding Number of Filters = No. of Input Channels x No. of Output Channels. No. of Params = Filter Size (F x F) * No. of Filters AlexNet [2012] - The pioneering paper.     - IP = 227 x 227 x 3     - First use of ReLU     - Used Norm Layers (not common anymore)     - Heavy Data Augmentation - Flipping, Jittering, Cropping, Color Normalization, etc     - Dropout = 0.5     - Batch Size = 128     - SGD Momentum = 0.9     - Learning rate 1e-2 reduced by 10 manually when val accuracy plateaus     - L2 weight decay 5e-4     - 7 CNN ensemble: 18.2% -> 15.4% Training Multiple Models and Averaging them together. ZF Net [2013] - Slight modif...

Default Activation Choice: ReLU Weights Initialization: Too Small - As we keep multiplying over and over, the activations will diminish. Too Large - multiply over and over again and explode. Default Initialization: Xaiver or MSRA. Zero Center and Normalize Layers - sensitivity will be high and generalization low if you don't. Sensitivity - Loss function change on changing params. So if less sensitive, optimization easier. Batch Normalization - Intermediate Activations to Zero Mean and Unit Variance - No of means, variance = number of channels, meaning mean over all batch over all data except channels. If validation plateaus - and training loss decreases - maybe you are overfitting - so add some regularization - like dropout. Low learning rate in theory should give better results if you train for large epochs, but a lot of time - so careful. Stuck in local minima - low learning rate - theoritically a problem - yes but practically- no. Today: Stochastic Gradient De...

Input Image: 224 x 224 x 3 All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225]. For all layers: Bias = True, Filter size = 3x3. Stride = 1, Padding = 1 (same) Each conv layer is followed by ReLU 19 layers are including 3 fully connected layers. So there are 16 convolutional layers with relu, 5 MaxPool Layers, 3 Fully connected layers. So, 16 kernels and 16 biases for conv layers and 3 kernels/weights and 3 biases for FC layers. 5 max pools (2x2) so output size before FC layers = 224 / (2^5) = 224/32 = 7 Also, Torch Input = C x H x W = 3 x 224 x 224 Conv2D(input_channel, output_channel, stride, padding) ---------------------------------------------------------------- Layer (type) Outpu...