wsl enable from windows features
wsl enter in command prompt
WSL 2 requires an update to its kernel component. For information please visit https://aka.ms/wsl2kernel
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
wsl --set-default-version 2
Still got: WSL 2 requires an update to its kernel component. For information please visit https://aka.ms/wsl2kernel
wsl --update
Failed to attach disk 'X:\WSL_IMPORT\ubuntu\ext4.vhdx' to WSL2: The system cannot find the path specified.
Error code: Wsl/Service/CreateInstance/MountVhd/HCS/ERROR_PATH_NOT_FOUND
wsl --unregister ubuntu
wsl --install
sudo apt install gcc [Didn't help but installed first]
sudo apt install g++ [Helped] - for detectron2
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
sudo apt-get install libgl1
import tensorflow as tf
print(len(tf.config.list_physical_devices('GPU')))
from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())
Note: Installing CUDA and CUDNN on WSL2 is same as normal Linux - but no Drivers required - as already installed on Windows - but make sure to download CUDA for WSL2 from NVIDIA's link.
Install cuda:
https://www.tensorflow.org/install/source#gpu
https://developer.nvidia.com/cuda-11-8-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=runfile_local
Find your cuda version here: https://www.tensorflow.org/install/source#gpu
wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.runsudo sh cuda_11.8.0_520.61.05_linux.runJust install CUDA others are not required. Below is just the information after setup, no need to do anything here setting LD path is enough see further below.
Driver: Not Selected
Toolkit: Installed in /usr/local/cuda-11.8/
Please make sure that
- PATH includes /usr/local/cuda-11.8/bin
- LD_LIBRARY_PATH includes /usr/local/cuda-11.8/lib64, or, add /usr/local/cuda-11.8/lib64 to /etc/ld.so.conf and run ldconfig as root
Then add this to .bashrc and source ~/.bashrc:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.8/lib64
Alternative option:
rm -rf -p $CONDA_PREFIX/etc/conda/activate.d
mkdir -p $CONDA_PREFIX/etc/conda/activate.d
echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.8/lib64' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
CUDNN is required otherwise still device not found python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"from tensorflow.python.client import device_libprint(device_lib.list_local_devices())
To see build info of current tensorflow:tf.sysconfig.get_build_info()import torchtorch.cuda.is_available()
Installing CUDNN:tar xf cudnn-linux-x86_64-8.6.0.163_cuda11-archive.tar.xz
cd cudnn-linux-x86_64-8.6.0.163_cuda11-archive/
sudo cp include/cudnn.h /usr/local/cuda-11.8/include
sudo cp lib/libcudnn* /usr/local/cuda-11.8/lib64
sudo chmod a+r /usr/local/cuda-11.8/include/cudnn.h /usr/local/cuda-11.8/lib64/libcudnn*See: https://algidus.blogspot.com/2022/11/installing-nvidia-driver-ubuntu-victus.htmlThe NUMA information/warning is a non-harmful warning. Tensorflow can run correctly with the warning.
Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory
*** SIGABRT received at time=1711473588 on cpu 4 ***
PC: @ 0x7f833a3989fc (unknown) pthread_kill
@ 0x7f833a344520 (unknown) (unknown)
[2024-03-26 11:19:48,737 E 394137 394873] logging.cc:361: *** SIGABRT received at time=1711473588 on cpu 4 ***
[2024-03-26 11:19:48,737 E 394137 394873] logging.cc:361: PC: @ 0x7f833a3989fc (unknown) pthread_kill
[2024-03-26 11:19:48,737 E 394137 394873] logging.cc:361: @ 0x7f833a344520 (unknown) (unknown)
ldconfig -p | grep libcud
I see I had libcuda.so.1 (libc6,x86-64) => /usr/lib/wsl/lib/libcuda.so.1
so I created a symbolic link
Actually I had it already so just added /usr/lib/wsl/lib/ to LD Library in bashrc and close and open terminal
python3 -c "import tensorflow as tf"
wget http://developer.download.nvidia.com/compute/cuda/10.0/Prod/patches/1/cuda_10.0.130.1_linux.run
cd cuda
sudo cp -P include/cudnn.h /usr/local/cuda-10.0/include
sudo cp -P lib64/libcudnn* /usr/local/cuda-10.0/lib64
sudo chmod a+r /usr/local/cuda-10.0/include/cudnn.h /usr/local/cuda-10.0/lib64/libcudnn*python3 -c "import tensorflow as tf"
You will see warnings like:/home/rb/miniconda3/envs/refml/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'._np_qint8 = np.dtype([("qint8", np.int8, 1)])first reboot: wsl.exe --shutdown and open vscode againTo supress use lower numpy version [ not needed though]python -m pip install numpy==1.16.4Sometimes you might face protobuf error:do: python -m pip install protobuf==3.20.* [already done in requirements for ReferenceMLTraining]
==============================ARCHIVE=========================================
Why I didn't use WSL much: WSL2 is too slow - same command runs faster on native terminal
- The reson could be because trying to run on /mnt/drive - on the default ext4 drive woks fine
Remove WSL:
>wsl --unregister ubuntu
followed by
>wsl --install
fixes the problem for ubuntu atleast
IF cc file not found error:
sudo apt update
sudo apt install build-essential
sudo apt-get install manpages-dev
https://www.tensorflow.org/install/source#gpu
https://www.tensorflow.org/install/pip
conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0add this to ~/.bashrcexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/
still not found on restart so had to do:cv2 error:
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
sudo apt-get update && sudo apt-get install ffmpeg libsm6 libxext6 -y
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib64/:/usr/lib/x86_64-linux-gnu/:$CONDA_PREFIX/lib/:/root/miniconda3/envs/automl/lib/python3.10/site-packages/nvidia/cuda_runtime/lib/:/root/miniconda3/envs/automl/lib/ #lib64 added to fix https://github.com/rstudio/rstudio/issues/8278# conda install -c conda-forge cudatoolkit=11.2
# pip install nvidia-cudnn-cu11=8.1.0
Install MiniConda (smaller / cli version of anaconda - not using gui of anaconda anyways)
- I think it's slow if windows directory has anaconda if inside wsl itself its fast
too many symbolic links - https://github.com/conda/conda/issues/10333
this appears to happen when you install conda in a windows location, rather than a pure WSL location. You can try uninstalling miniconda, and then reinstalling in your WSL home directory (by running cd after loading the WSL shell
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
example: conda info --envs
Installing aws on WSL2
sudo apt update -y && sudo apt upgrade -y [Important]
sudo apt install glibc-source groff less unzip -y
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
AWS Credentials :
set this in ~/.bashrc
# AWS CONFIG SETUP
export AWS_CONFIG_FILE=/mnt/x/.aws/config
export AWS_SHARED_CREDENTIALS_FILE=/mnt/x/.aws/credentials
ID_RSA didn't work with windows drive - permission issues so moved to root
https://superuser.com/questions/1161247/wsl-openssh-ignores-private-key-which-is-stored-in-windows-mnt-c-how-to-chan
Won't work on windows drive because: Windows will always translate the permissions of the file to those common in Windows.
cp .ssh/id_rsa /root/.ssh/
chmod 600 id_rsa
eval "$(ssh-agent -s)"
ssh-add id_rsa
https://dev.to/mefaba/installing-wsl-on-another-drive-in-windows-5c4a
Check available distros:
wsl.exe --list --online
PS C:\Users\rb> wsl --list -v
NAME STATE VERSION
* Ubuntu-18.04 Stopped 2
PS C:\Users\rb> wsl -t Ubuntu
There is no distribution with the supplied name.
PS C:\Users\rb> wsl -t Ubuntu-18.04
PS C:\Users\rb> wsl --list -v
NAME STATE VERSION
* Ubuntu-18.04 Stopped 2
PS C:\Users\rb> wsl --export Ubuntu-18.04 E:\wsl_export\ubuntu-ex.tar
The system cannot find the path specified.
PS C:\Users\rb> wsl --export Ubuntu-18.04 E:\wsl_export\ubuntu-ex.tar
PS C:\Users\rb> wsl --unregister Ubuntu-18.04
Unregistering...
PS C:\Users\rb> wsl --list -v
Windows Subsystem for Linux has no installed distributions.
Distributions can be installed by visiting the Microsoft Store:
https://aka.ms/wslstore
PS C:\Users\rb> wsl --import Ubuntu-18.04 "E:\wsl_import\ubuntu" "E:\wsl_export\ubuntu-ex.tar"
PS C:\Users\rb> wsl --list -v
NAME STATE VERSION
* Ubuntu-18.04 Stopped 2
PS C:\Users\rb>
NPM Setup:
https://medium.com/@lucaskay/install-node-and-npm-using-nvm-in-mac-or-linux-ubuntu-f0c85153e173
https://github.com/nvm-sh/nvm#installing-and-updating
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
restart terminal
nvm --version
0.39.3
# to remove: nvm deactivate
[npm and node not installed yet]
nvm install node
node --version
npm --version
USe this: nvm use 16.16.0
Other version failed
(base) root@Optimus:/mnt/x# node --version
v20.3.1
(base) root@Optimus:/mnt/x# npm --version
9.6.7
nvm use node
//to install a specific version number
nvm install 8.9 //to install node 8.9
nvm use 8.9
//to install the LTS version
nvm install --lts
nvm use --lts
nvm use desired-version
Now make sure npm was installed by running
npm -v