Geospatial python tutorial - PyQGIS in Jupyter Notebook

I did a lot of my PyQGIS scripting in QGIS python console and also PyCharm for plugins development. But I wanted something different this time, so tried to use PyQGIS in Jupyter notebook. I have been facing some issues to run PyQGIS inside Jupyter Notebook before. Since I already resolved that issues, I am happy to share it and hopefully it is really useful for your awesome projects.

It is fun to do some experiment on this and I hope you guys can build something wonderful in Jupyter using QGIS libraries and module. Well, at least that is my intention.

Before we start writing our script in Jupyter Notebook , we need to setup our environment by creating a *.cmd or *.bat file.

Setup

Setting the Environment

First, we need to install Jupyter notebook in QGIS3 environment, please check my previous post on how to install it.

Before we begin our scripting in Jupyter, you might want to copy and paste batch script below and save it as *.cmd or *.bat. Make sure you check the directory, edit the path or directory wherever possible. At the bottom line there is a line called call jupyter notebook, you can change it to cmd.exe if you got something in your mind and need to use command line.

Notes ! : Please check all the directories, make sure it is available. I am using Windows, so it could be different in other OS system. I am also using OSGEO4W for my QGIS installation, so please use your QGIS installation directory.

@echo off
call "%~dp0\o4w_env.bat"
call "%OSGEO4W_ROOT%\apps\grass\grass76\etc\env.bat"
call qt5_env.bat
call py3_env.bat
@echo off
path %OSGEO4W_ROOT%\apps\qgis\bin;%OSGEO4W_ROOT%\apps\grass\grass76\lib;%OSGEO4W_ROOT%\apps\grass\grass76\bin;%PATH%
PATH %OSGEO4W_ROOT%\apps\qgis\python;%PATH%
PATH %OSGEO4W_ROOT%\apps\qgis\bin;%PATH%
PATH %OSGEO4W_ROOT%\apps\Qt5\bin;%PATH%
PATH %OSGEO4W_ROOT%\apps\Python37;%PATH%
PATH %OSGEO4W_ROOT%\apps\Python37\Scripts;%PATH%
PATH %OSGEO4W_ROOT%\apps\qgis\python\plugins;%PATH%
PATH %OSGEO4W_ROOT%\apps\Python37\DLLs;%PATH%
PATH %OSGEO4W_ROOT%\apps\Python37\lib;%PATH%
PATH %OSGEO4W_ROOT%\apps\Python37\lib\site\packages;%PATH%

set QGIS_PREFIX_PATH=%OSGEO4W_ROOT:\=/%/apps/qgis
set GDAL_FILENAME_IS_UTF8=YES
set VSI_CACHE=TRUE
set VSI_CACHE_SIZE=1000000
set QT_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\qgis\qtplugins;%OSGEO4W_ROOT%\apps\qt5\plugins
set PYTHONPATH=%OSGEO4W_ROOT%\apps\qgis\python

@echo off
call jupyter notebook

Save this file somewhere in you computer, or just save it in the E:\OSGeo4W64\bin so you can run it from OSGEO4W Shell. Name it as run-jupyter.bat so whenever we open OSGeo4W Shell just type run-jupyter.bat

Testing our script in Jupyter Notebook!

In jupyter notebook, we can start writing our script..

Try to write something as below, if there are no error messages, then you are doing great.

from qgis.core import *

So now, lets try to do something simple, like extracting vertices from vector data.

import sys
import os
from osgeo import ogr
from qgis.core import (
     QgsApplication,
     QgsProcessingFeedback,
     QgsVectorLayer
)
from qgis.analysis import QgsNativeAlgorithms

We need python to set the QGIS prefix, and access it from E:\OSGeo4W64\apps\qgis. After we initiate QGIS in python system, append the plugins libraries by accessing this folder, E:\OSGeo4W64\apps\qgis\python\plugins.

QgsApplication.setPrefixPath(r'E:\OSGeo4W64\apps\qgis', True)
qgs = QgsApplication([], False)
qgs.initQgis()
sys.path.append(r'E:\OSGeo4W64\apps\qgis\python\plugins')

We need to know all the available processing module within QGIS, so follow as bellow.

import geopandas as gpd
import matplotlib.pyplot as plt
plt.ion()

import processing
from processing.core.Processing import Processing
Processing.initialize()
QgsApplication.processingRegistry().addProvider(QgsNativeAlgorithms())
for alg in QgsApplication.processingRegistry().algorithms():
        print(alg.id(), "--->", alg.displayName())

Set our source data, then plot using matplotlib.

source = r"E:\Program_dev\data\test.geojson"
inputvector = QgsVectorLayer(r"E:\Program_dev\data\test.geojson")
fig, ax = plt.subplots(figsize = (10,10))
gpd.read_file(source).plot(ax=ax);

png

Extract Vertices

Before we start, lets check the parameter need to run this algorithm by following below scripts

processing.algorithmHelp("native:extractvertices")

vertice_out = r"E:\Program_dev\data\out_vertices.geojson"
# just delete the output if exist
if os.path.exists(vertice_out):
    os.remove(vertice_out)
else:
    pass
params = {
    'INPUT': inputvector,
    'OUTPUT': vertice_out
}
feedback = QgsProcessingFeedback()
processing.run("native:extractvertices", params, feedback=feedback)
fig, ax = plt.subplots(figsize = (10,10))
gpd.read_file(vertice_out).plot(ax=ax);

png

Buffer

processing.algorithmHelp("native:buffer")

buf_out = r"E:\Program_dev\data\buffer_output.geojson"
# just delete the output if exist
if os.path.exists(buf_out):
    os.remove(buf_out)
else:
    pass
params = {
    'INPUT': inputvector,
    'DISTANCE': 50,
    'SEGMENTS': 5,
    'END_CAP_STYLE':0,
    'JOIN_STYLE':0,
    'MITER_LIMIT':2,
    'DISSOLVE': False,
    'OUTPUT': buf_out
}
feedback = QgsProcessingFeedback()
processing.run("native:buffer", params, feedback=feedback)
# plot the data using geopandas .plot() method
fig, ax = plt.subplots(figsize = (10,10))
gpd.read_file(buf_out).plot(ax=ax);

png

Random points

processing.algorithmHelp("qgis:randompointsinsidepolygons")

random_out = r"E:\Program_dev\data\random_output.geojson"
# just delete the output if exist
if os.path.exists(buf_out):
    os.remove(buf_out)
else:
    pass
params = {
    'INPUT': inputvector,
    'STRATEGY': 0,
    'EXPRESSION': 20,
    'MIN_DISTANCE':1,
    'OUTPUT': random_out
}
feedback = QgsProcessingFeedback()
processing.run("qgis:randompointsinsidepolygons", params, feedback=feedback)
fig, ax = plt.subplots(figsize = (10,10))
gpd.read_file(random_out).plot(ax=ax);

png

Have fun exploring QGIS using Jupyter Notebook!

Lerry William S.