wsl enable from windows features
wsl enter in command prompt
WSL 2 requires an update to its kernel component. For information please visit https://aka.ms/wsl2kernel
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
wsl --set-default-version 2
Still got: WSL 2 requires an update to its kernel component. For information please visit https://aka.ms/wsl2kernel
wsl --update
Failed to attach disk 'X:\WSL_IMPORT\ubuntu\ext4.vhdx' to WSL2: The system cannot find the path specified.
Error code: Wsl/Service/CreateInstance/MountVhd/HCS/ERROR_PATH_NOT_FOUND
wsl --unregister ubuntu
wsl --install
sudo apt install gcc [Didn't help but installed first]
sudo apt install g++ [Helped] - for detectron2
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
sudo apt-get install libgl1
import tensorflow as tf
print(len(tf.config.list_physical_devices('GPU')))
from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())
Note: Installing CUDA and CUDNN on WSL2 is same as normal Linux - but no Drivers required - as already installed on Windows - but make sure to download CUDA for WSL2 from NVIDIA's link.
Install cuda:
https://www.tensorflow.org/install/source#gpu
https://developer.nvidia.com/cuda-11-8-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=runfile_local
Find your cuda version here: https://www.tensorflow.org/install/source#gpu
wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.runsudo sh cuda_11.8.0_520.61.05_linux.runJust install CUDA others are not required. Below is just the information after setup, no need to do anything here setting LD path is enough see further below.
Driver: Not Selected
Toolkit: Installed in /usr/local/cuda-11.8/
Please make sure that
- PATH includes /usr/local/cuda-11.8/bin
- LD_LIBRARY_PATH includes /usr/local/cuda-11.8/lib64, or, add /usr/local/cuda-11.8/lib64 to /etc/ld.so.conf and run ldconfig as root
Then add this to .bashrc and source ~/.bashrc:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.8/lib64
Alternative option:
rm -rf -p $CONDA_PREFIX/etc/conda/activate.d
mkdir -p $CONDA_PREFIX/etc/conda/activate.d
echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.8/lib64' >> $CONDA_PREFIX/etc/conda/activate.d/env_vars.sh
CUDNN is required otherwise still device not found python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"from tensorflow.python.client import device_libprint(device_lib.list_local_devices())
To see build info of current tensorflow:tf.sysconfig.get_build_info()import torchtorch.cuda.is_available()
Installing CUDNN:tar xf cudnn-linux-x86_64-8.6.0.163_cuda11-archive.tar.xz
cd cudnn-linux-x86_64-8.6.0.163_cuda11-archive/
sudo cp include/cudnn.h /usr/local/cuda-11.8/include
sudo cp lib/libcudnn* /usr/local/cuda-11.8/lib64
sudo chmod a+r /usr/local/cuda-11.8/include/cudnn.h /usr/local/cuda-11.8/lib64/libcudnn*See: https://algidus.blogspot.com/2022/11/installing-nvidia-driver-ubuntu-victus.htmlThe NUMA information/warning is a non-harmful warning. Tensorflow can run correctly with the warning.
Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory
*** SIGABRT received at time=1711473588 on cpu 4 ***
PC: @ 0x7f833a3989fc (unknown) pthread_kill
@ 0x7f833a344520 (unknown) (unknown)
[2024-03-26 11:19:48,737 E 394137 394873] logging.cc:361: *** SIGABRT received at time=1711473588 on cpu 4 ***
[2024-03-26 11:19:48,737 E 394137 394873] logging.cc:361: PC: @ 0x7f833a3989fc (unknown) pthread_kill
[2024-03-26 11:19:48,737 E 394137 394873] logging.cc:361: @ 0x7f833a344520 (unknown) (unknown)
ldconfig -p | grep libcud
I see I had libcuda.so.1 (libc6,x86-64) => /usr/lib/wsl/lib/libcuda.so.1
so I created a symbolic link
Actually I had it already so just added /usr/lib/wsl/lib/ to LD Library in bashrc and close and open terminal
python3 -c "import tensorflow as tf"
wget http://developer.download.nvidia.com/compute/cuda/10.0/Prod/patches/1/cuda_10.0.130.1_linux.run
cd cuda
sudo cp -P include/cudnn.h /usr/local/cuda-10.0/include
sudo cp -P lib64/libcudnn* /usr/local/cuda-10.0/lib64
sudo chmod a+r /usr/local/cuda-10.0/include/cudnn.h /usr/local/cuda-10.0/lib64/libcudnn*python3 -c "import tensorflow as tf"
You will see warnings like:/home/rb/miniconda3/envs/refml/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'._np_qint8 = np.dtype([("qint8", np.int8, 1)])first reboot: wsl.exe --shutdown and open vscode againTo supress use lower numpy version [ not needed though]python -m pip install numpy==1.16.4Sometimes you might face protobuf error:do: python -m pip install protobuf==3.20.* [already done in requirements for ReferenceMLTraining]
==============================ARCHIVE=========================================
Why I didn't use WSL much: WSL2 is too slow - same command runs faster on native terminal
- The reson could be because trying to run on /mnt/drive - on the default ext4 drive woks fine
Remove WSL:
>wsl --unregister ubuntu
followed by
>wsl --install
fixes the problem for ubuntu atleast
IF cc file not found error:
sudo apt update
sudo apt install build-essential
sudo apt-get install manpages-dev
https://www.tensorflow.org/install/source#gpu
https://www.tensorflow.org/install/pip
conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0add this to ~/.bashrcexport LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/
still not found on restart so had to do:cv2 error:
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
sudo apt-get update && sudo apt-get install ffmpeg libsm6 libxext6 -y
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib64/:/usr/lib/x86_64-linux-gnu/:$CONDA_PREFIX/lib/:/root/miniconda3/envs/automl/lib/python3.10/site-packages/nvidia/cuda_runtime/lib/:/root/miniconda3/envs/automl/lib/ #lib64 added to fix https://github.com/rstudio/rstudio/issues/8278# conda install -c conda-forge cudatoolkit=11.2
# pip install nvidia-cudnn-cu11=8.1.0
Install MiniConda (smaller / cli version of anaconda - not using gui of anaconda anyways)
- I think it's slow if windows directory has anaconda if inside wsl itself its fast
too many symbolic links - https://github.com/conda/conda/issues/10333
this appears to happen when you install conda in a windows location, rather than a pure WSL location. You can try uninstalling miniconda, and then reinstalling in your WSL home directory (by running cd after loading the WSL shell
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
example: conda info --envs
Installing aws on WSL2
sudo apt update -y && sudo apt upgrade -y [Important]
sudo apt install glibc-source groff less unzip -y
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
AWS Credentials :
set this in ~/.bashrc
# AWS CONFIG SETUP
export AWS_CONFIG_FILE=/mnt/x/.aws/config
export AWS_SHARED_CREDENTIALS_FILE=/mnt/x/.aws/credentials
ID_RSA didn't work with windows drive - permission issues so moved to root
https://superuser.com/questions/1161247/wsl-openssh-ignores-private-key-which-is-stored-in-windows-mnt-c-how-to-chan
Won't work on windows drive because: Windows will always translate the permissions of the file to those common in Windows.
cp .ssh/id_rsa /root/.ssh/
chmod 600 id_rsa
eval "$(ssh-agent -s)"
ssh-add id_rsa
https://dev.to/mefaba/installing-wsl-on-another-drive-in-windows-5c4a
Check available distros:
wsl.exe --list --online
PS C:\Users\rb> wsl --list -v
NAME STATE VERSION
* Ubuntu-18.04 Stopped 2
PS C:\Users\rb> wsl -t Ubuntu
There is no distribution with the supplied name.
PS C:\Users\rb> wsl -t Ubuntu-18.04
PS C:\Users\rb> wsl --list -v
NAME STATE VERSION
* Ubuntu-18.04 Stopped 2
PS C:\Users\rb> wsl --export Ubuntu-18.04 E:\wsl_export\ubuntu-ex.tar
The system cannot find the path specified.
PS C:\Users\rb> wsl --export Ubuntu-18.04 E:\wsl_export\ubuntu-ex.tar
PS C:\Users\rb> wsl --unregister Ubuntu-18.04
Unregistering...
PS C:\Users\rb> wsl --list -v
Windows Subsystem for Linux has no installed distributions.
Distributions can be installed by visiting the Microsoft Store:
https://aka.ms/wslstore
PS C:\Users\rb> wsl --import Ubuntu-18.04 "E:\wsl_import\ubuntu" "E:\wsl_export\ubuntu-ex.tar"
PS C:\Users\rb> wsl --list -v
NAME STATE VERSION
* Ubuntu-18.04 Stopped 2
PS C:\Users\rb>
NPM Setup:
https://medium.com/@lucaskay/install-node-and-npm-using-nvm-in-mac-or-linux-ubuntu-f0c85153e173
https://github.com/nvm-sh/nvm#installing-and-updating
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
restart terminal
nvm --version
0.39.3
# to remove: nvm deactivate
[npm and node not installed yet]
nvm install node
node --version
npm --version
USe this: nvm use 16.16.0
Other version failed
(base) root@Optimus:/mnt/x# node --version
v20.3.1
(base) root@Optimus:/mnt/x# npm --version
9.6.7
nvm use node
//to install a specific version number
nvm install 8.9 //to install node 8.9
nvm use 8.9
//to install the LTS version
nvm install --lts
nvm use --lts
nvm use desired-version
Now make sure npm was installed by running
npm -v
Move WSL to another drive:
I have found an approach that does not require deleting (unregistering) and importing an existing distibution.
Stop the distribution you want to relocate:
wsl --terminate Ubuntuorwsl --shutdown.Create a backup, just to be sure:
wsl --export Ubuntu D:\Ubuntu-backup.tar.Start the Registry Editor (
regedit.exe- it requires elevated permissions).Navigate to
Computer\HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Lxss\. Here, you can find all the existing distributions. Find the one that you want to relocate based on theDistributionNameentry (in my case, it isUbuntu).The
BasePathentry shows the current location of the disk. In my case:C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState. If you open this folder, you will see that it has oneext4.vhdxfile. This is the disk file.Copy this file to the location where you want to have it. For example,
cp C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\ext4.vhdx D:\wsl\Ubuntu\.Change the
BasePathentry in Registry Editor to point to the new location of theext4.vhdx.Just start the WSL distribution, it will use the disk in the new location. At this point, you can delete the disk file from the original location.