Setup and Integrate Colab + Google Drive for your Data Science Project
I recently worked on a data science project that required handling very large datasets and processor power. I wanted a private cloud environment to host and execute my project, so I decided to try Google Colabs and in this article, I will be showing you how to do the same.
- Create a Notebook on Colab
There are many ways to get started on Google Colab. You can upload an existing notebook from your local drive, use examples on Colab, get files from your google drive, or clone from your GitHub repo. However, in this tutorial, we will be creating a new empty notebook.
2. Mount Your Google Drive
The next step is to link your Google Drive to Colabs. This will make it easy for you to manage your projects, data, and models.
Type this code into a new cell and run. This will require you to authorize allow Colab to gain access to Google drive. Click the link, copy and paste the generated code to mount your Google Drive on the current Colab Notebook Project.
3. Download/ Clone Project Files
Clone GitHub Repo
If you would like to clone a repo from your account, you can use the code below. Please bear in mind that you will have to encode special characters like “@” to “%40” to avoid receiving error messages.
!git clone https://example%40gmail.com:password@github.com/nnitiwe-dev/Resume_classifier.git
Or you can simply clone public Repos with the code:
!git clone https://github.com/nnitiwe-dev/Resume_classifier.git
Copy a shared Google Folder
In other to Download from a Shared Google folder, you will have to modify the link below to include the shared file Id:
for instance:
Next, download the “gdown” python library which will help you download. Once you have downloaded the file, you can unzip the content (in the case of a zipped folder) in the desired Google Drive location:
!gdown https://drive.google.com/uc?id=a1OwHlNw4
!unzip downloaded_file.zip -d content/drive/MyDrive/DESIRED_PATH/
Download File from Public URL
Use wget to download files from public URLs to the desired folder:
!wget -O ‘/content/drive/MyDrive/DESIRED_PATH/filename’ ‘https://PUBLIC_URL_TO_FILE’
4. Navigating and creating Environments
To get the URL of any file/folder on your Google drive, simply navigate to the specific file and click options>copy path.
Whenever you want to navigate between folders via the terminal, use the notebook magic command:
%cd FOLDER_URL
Creating an Environment with Conda
Miniconda is a free minimal installer for conda. It is a small, bootstrap version of Anaconda that can be used on Google Colabs. To install miniconda and create environments, follow the instructions below:
Install miniconda
%%bash
MINICONDA_INSTALLER_SCRIPT=Miniconda3–4.5.4-Linux-x86_64.sh
MINICONDA_PREFIX=/usr/local
wget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPT
chmod +x $MINICONDA_INSTALLER_SCRIPT
./$MINICONDA_INSTALLER_SCRIPT -b -f -p $MINICONDA_PREFIX
Create and Activate Environment
!conda create -n project-environ python=3.7 --yes
!activate project-environ
If you found this article helpful, I would appreciate your support 😁👏
