Ever since ChatGPT and OpenAI came out to the world for public access, developers around the world have been scrambling to create innovative applications and products that can take advantage of these new AI/ML capabilities. They have a brilliant idea to test with AI/ML, but provisioning and managing necessary infrastructure in the cloud is quite overwhelming. This is where PaaS service like App Service comes to the rescue. Recently, one of my colleagues approached me with a similar predicament — how to run a Python ML web app on App Service, which happens to be an OpenAI-based app. Then I realized the potential of App Services for AI/ML developers had significantly been overlooked by developers. By having a PaaS to quickly iterate ideas without the hassle of infrastructure management, App Service can give Python developers a significant competitive advantage in the era of artificial intelligence and machine learning.
Basics of Python on App Service
There are two essential points to remember when running Python on App Service. By understanding these points here, you’ll be able to tackle any issues that may arise in the future.
- App Service doesn’t build the application(
pip install)by default, meaning it expects the self-contained project in a Zip file. If you want to iterate only with the code, App Service has a build system called “Oryx” which can take your source code and build it upon deployment. - App Service uses containers under the hood, so any app on App Service (including Python) must listen on 0.0.0.0 to accept the traffic. Python image used by App Service expects port 8000 to be open. For the custom port,
PORTenvironment variable should be set accordingly.
Zip file deployment expects a specific name for the virtual environment, which is “antev”. Developers can supply the custom name set in VIRTUALENV_NAME environment variable. The default startup command in the base container image is gunicorn --bind=0.0.0.0 --timeout 600, so it doesn't listen on “0.0.0.0”. In case the app is not running on gunicorn, it requires to 1. listen on 0.0.0.0 and 2. custom startup command like “python app.py”. This command is sent to the runtime container image as below to start an app on App Service.
$ docker run -d --expose=8000 --name gradio-test_0_de2ec092 -e PORT=8000 -e WEBSITE_SITE_NAME=gradio-test -e WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_HOSTNAME=gradio-test.azurewebsites.net -e WEBSITE_INSTANCE_ID=d7187ed933f101a2d8831933f95ddafaf5f9f7f448215c7beccc82ff5d275fd3 -e HTTP_LOGGING_ENABLED=1 -e WEBSITE_USE_DIAGNOSTIC_SERVER=False appsvc/python:3.9_20230116.1.tuxprod python $APP_PATH/app.pyOryx Build Service
When it comes to deploying an application on App Service, you have several options to choose from, including a continuous deployment with Git using Deployment Center, ZipDeploy using the CLI (az webapp deploy), SFTP, and more. While Deployment Center uses Oryx by default to build the source code upon commits, other deployment options don’t trigger Oryx automatically. To activate Oryx during deployment, you’ll need to set two specific environment variables — SCM_DO_BUILD_DURING_DEPLOYMENT to true and ENABLE_ORYX_BUILD to true.
If you’re wondering how Oryx build service on App Service works, you could try it locally as Oryx build is basically run on a container image. Trying it locally will make you better understand how it works internally.
$ docker run --volume $(pwd):/repo 'mcr.microsoft.com/oryx/build:latest' oryx build /repo --platform python --platform-version 3.9.7 --output /repo
Operation performed by Microsoft Oryx, https://github.com/Microsoft/Oryx
You can report issues at https://github.com/Microsoft/Oryx/issues
Oryx Version: 0.2.20230501.1, Commit: 58513374525ab2977af68fc5b3c344d32858d8af, ReleaseTagName: 20230501.1
Build Operation ID: b2c7df87171a1a52
OS Type : stretch
Image Type : full
Detecting platforms...
Detected following platforms:
python: 3.9.7
Version '3.9.7' of platform 'python' is not installed. Generating script to install it...
Source directory : /repo
Destination directory: /repo
...
$ ls
antenv app.py requirements.txt oryx-manifest.toml You’ll notice two key outputs: oryx-manifest.toml file and the virtual environment antev. On App Service, the virtual environment antenv is compressed into a file called ‘output.tar.gz’ which is then placed at /home/site/wwwroot. At the time of startup on App Service, this file is automatically unpacked into a temporary folder in the runtime container for the seamless execution of your app.
Deploy ML web app on App Service
I have a sample ML web app that I took from Gradio’s example repo, which is a text-to-speech app. You can access the source code at https://github.com/eggboy/gradio-tts. For those unfamiliar with Gradio, Gradio is a versatile web framework that can be used to create GUI/demo around any machine learning model. Just like any other ML apps, this sample app requires several OS dependencies and also a custom startup command as it doesn’t run on gunicorn web server. Let’s create an App Service with a Linux plan first.
By default, App Service creates a domain that’s ready for you to access right away. When you first access it, you’ll see a simple app that’s served from the /opt/defaultsite folder, which displays a static HTML file at /home/site/wwwroot/hostingstart.html . This is pre-defined behavior set in the base container image for Python on App Service.
Before we deploy an app, we will configure two things 1. Oryx, and 2. Startup command. As discussed earlier, Oryx can be configured with two environment variables.
Here’s where things get interesting. Since App Service relies on container images, installing Linux package is not a trivial thing as developers don’t have the freedom to touch base images unless they deploy an app in their own container. on App Service, any packages not part of the base image should be installed with a startup command. Look at the command below; the startup command execute apt-get to install necessary Linux packages, then start an app using python $APP_PATH/app.py.
apt-get update -yy && apt-get install -yy libsndfile1 espeak-ng ffmpeg libavcodec-extra && python $APP_PATH/app.py
Saving the configuration will always trigger a restart of your App Service instance. Now, let’s move on to deploying the sample app. To do this, we’ll be using Azure CLI — first, we’ll need to zip our source files, and then use the az webapp deploy command. It can take some time to build your app, so be patient. One tip for a successful deployment is to use the async option as it takes more than the default timeout of approximately 5min.
$ zip app.zip *
adding: app.py (deflated 48%)
$ az webapp deploy --name gradio-test-tts --resource-group sandbox-rg --src-path ./app.zip --type zip --timeout 90000 --async true
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatusOnce deployment is done, open the default domain with a web browser. Early on, apt-get installed only espeak-ng, so it will only work with Language en.
Wrapping Up
In this article, I covered the core concepts of Python on App Service by deploying a sample app. With only a few configuration steps, the ML web app could run in minutes in the cloud. You should see the platform's value for developers, where they can quickly iterate their ideas without the hassle of managing infrastructure.
If you liked my article, please leave a few claps or start following me. You can get notified whenever I publish something new. Let’s stay connected on Linkedin, too! Thanks so much for reading!
