Category: DevOps

How to upgrade to Python 3.9.0 on Ubuntu 18.04 LTS without internal bleeding!

SPOILER ALERT:
Upgrading the Python version on Ubuntu breaks some packages like apt and pip in your system!
[Bleeding SysAdmin crawling]

In this article we’re going to upgrade python and fix the issues.

Start reading this article and executing the instruction:
How to upgrade to Python 3.9.0 on Ubuntu 18.04 LTS
It will guide you through updating the repo and basic updating stuff.

After that, try running sudo apt update.
See!

Traceback (most recent call last): File "/usr/lib/cnf-update-db", line 8, in from CommandNotFound.db.creator import DbCreator File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 11, in import apt_pkg ModuleNotFoundError: No module named 'apt_pkg'
ModuleNotFoundError: No module named ‘apt_pkg’

or event a simple pip3 install virtualenv:

Traceback (most recent call last): File "/usr/bin/pip3", line 9, in <module> from pip import main File "/usr/lib/python3/dist-packages/pip/__init__.py", line 29, in <module> from pip.utils import get_installed_distributions, get_prog File "/usr/lib/python3/dist-packages/pip/utils/__init__.py", line 23, in <module> from pip.locations import ( File "/usr/lib/python3/dist-packages/pip/locations.py", line 9, in <module> from distutils import sysconfig ImportError: cannot import name 'sysconfig' from 'distutils' (/usr/lib/python3.9/distutils/__init__.py)
ImportError: cannot import name ‘sysconfig’ from ‘distutils’ (/usr/lib/python3.9/distutils/__init__.py)

Well, that’s a lot of errors!
BUT:
This is GNU/Linux bruh! We always have solutions! [D’Oh!]

At first, you need to resync old python packages to new corresponding paths, we’ll first navigate to python3 dist-packages folder and use ln command to make links.

cd /usr/lib/python3/dist-packages sudo ln -s apt_pkg.cpython-{36m,39m}-x86_64-linux-gnu.so
Linking old python packages to new ones

If you consider your old python version 3.6 and the new one 3.9 the ln command would be something like above: (change 36 and 39 numbers accordingly)

Then uninstall any existing python-apt packages using two apt purge commands and reinstall them just like a normal apt package

Reinstalling python-apt package

BEWARE: using apt with --reinstall flag doesn’t work!
Now run sudo apt update.

YaY! It’s fixed! [Mic Drop!]

But wait a second, I still can’t install pip packages!
Just run this command and you’ll be good to go:
sudo apt install python3.9-distutils python3.9-dev

Happy SysAdminning!

Thanks to the free software community for this beautiful world we’re living in!

Sources:
How to upgrade to Python 3.9.0 on Ubuntu 18.04 LTS
StackOverFlow: python-dev installation error: ImportError: No module named apt_pkg
AskUbuntu: ImportError: cannot import name ‘sysconfig’ from ‘distutils’ (/usr/lib/python3.9/distutils/__init__.py)

https://stackoverflow.com/questions/13708180/python-dev-installation-error-importerror-no-module-named-apt-pkg

Serverless Micro Django | Lightweight, yet powerful Python utility for lambda functions

Serverless Micro Django
https://github.com/mmoallemi99/serverless-micro-django/

A couple of weeks ago I had this task in Agileful to refactor and improve some AWS lambda functions.

There was a bunch of pure SQL queries written alongside python codes which made me think, what if we could use standalone Django ORM in our python lambda functions?

So I did some digging on how we can use my beloved ORM in python scripts despite the serverless stuff, there were some vague and old answers but I managed to make something out of it.
It’s as simple as defining a database connection settings and calling django.setup().

Standalone Django ORM

Here is the minimal python code needed to use standalone Django ORM:

Python Codes Pt. 1
handler.py
Serverless Micro Django - Python Codes Pt. 2
serverless_micro_django/settings.py

Plus these two files you need to create your desired Django app models. (e.g. videos_app/models.py)

There is one downside in this way of using Django and that is the lack of manage.py !
I haven’t got time to try adding manage.py commands so I added some helper scripts like the ones below:

Serverless Micro Django - Python Codes Pt. 3
makemigrations.py / migrate.py

So even if you just need Django ORM without all the serverless and fancy stuff use the codes above and you’re good to go. Or check out the GitHub repository:
https://github.com/mmoallemi99/serverless-micro-django

Serverless Micro Django!

So let’s make it Serverless!
As we know lambda functions must accept two function parameters: event, context.

As SMD (Serverless Micro Django) is supposed to be lightweight and micro! it doesn’t support any REST API like Django & Flask. Just function!

Hence when you are invoking the function you must pass your desired action/event in lambda event argument:

SMD - Python Codes Pt. 4
test_lambda.py

And in the end, I developed a ModelController so I can execute the events. Which I’m not going to get into and talk about.

I’ve also done some event input validations and modeling using PyDantic which you can find in the project’s GitHub repository.

I know there is a long way for SMD to become mature and production-ready. I would gladly accept and be open to contributions and feedback. (Documenting, Development, Content Writing, etc…)

GitHub Repository:
https://github.com/mmoallemi99/serverless-micro-django