Instance status:
SystemSetup: Succeeded
UserContainerImagePull: Succeeded
ModelDownload: Succeeded
UserContainerStart: InProgress

Container events:
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:49:19.644771Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:49:22.275633Z, Message: Readiness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:49:29.496341Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:49:30.765522Z, Message: Readiness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:49:39.496115Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:49:40.76566Z, Message: Readiness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:49:50.809232Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:49:50.810784Z, Message: Readiness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:49:59.496228Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:00.766082Z, Message: Readiness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:09.575839Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:10.765608Z, Message: Readiness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:19.496038Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:20.765557Z, Message: Readiness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:29.495899Z, Message: Liveness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:30.765722Z, Message: Readiness probe failed: HTTP probe failed with statuscode: 502
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:39.495585Z, Message: Liveness probe failed: Get http://10.66.0.2:5001/: dial tcp 10.66.0.2:5001: connect: connection refused
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:40.769494Z, Message: Readiness probe failed: Get http://10.66.0.2:5001/: dial tcp 10.66.0.2:5001: connect: connection refused
Kind: Pod, Name: LivenessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:49.49559Z, Message: Liveness probe failed: Get http://10.66.0.2:5001/: dial tcp 10.66.0.2:5001: connect: connection refused
Kind: Pod, Name: ReadinessProbeFailed, Type: Warning, Time: 2022-11-02T08:50:50.765264Z, Message: Readiness probe failed: Get http://10.66.0.2:5001/: dial tcp 10.66.0.2:5001: connect: connection refused

Container logs:
2022-11-02T08:47:46,589823422+00:00 - nginx/run 
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:1
2022-11-02T08:47:46,595131969+00:00 - gunicorn/run 
2022-11-02T08:47:46,598711033+00:00 | gunicorn/run | 
2022-11-02T08:47:46,599886621+00:00 | gunicorn/run | ###############################################
2022-11-02T08:47:46,601126609+00:00 | gunicorn/run | AzureML Container Runtime Information
2022-11-02T08:47:46,602339297+00:00 | gunicorn/run | ###############################################
2022-11-02T08:47:46,603612184+00:00 | gunicorn/run | 
2022-11-02T08:47:46,605063069+00:00 | gunicorn/run | 
2022-11-02T08:47:46,607558444+00:00 | gunicorn/run | AzureML image information: mlflow-ubuntu18.04-py37-cpu-inference:20220926.v1
2022-11-02T08:47:46,608774532+00:00 | gunicorn/run | 
2022-11-02T08:47:46,610066519+00:00 | gunicorn/run | 
2022-11-02T08:47:46,611451905+00:00 | gunicorn/run | PATH environment variable: /opt/miniconda/envs/amlenv/bin:/opt/miniconda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2022-11-02T08:47:46,612783292+00:00 | gunicorn/run | PYTHONPATH environment variable: 
2022-11-02T08:47:46,614124579+00:00 | gunicorn/run | 
2022-11-02T08:47:47,632541216+00:00 | gunicorn/run | CONDAPATH environment variable: /opt/miniconda

# conda environments:
#
base                  *  /opt/miniconda
amlenv                   /opt/miniconda/envs/amlenv

2022-11-02T08:47:47,962110168+00:00 | gunicorn/run | 
2022-11-02T08:47:47,963600869+00:00 | gunicorn/run | Pip Dependencies (before dynamic installation)

azure-core==1.25.1
azure-identity==1.11.0
azureml-inference-server-http==0.7.6
cachetools==5.2.0
certifi==2022.9.24
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.3
cloudpickle==2.2.0
cryptography==38.0.1
databricks-cli==0.17.3
entrypoints==0.4
Flask==2.1.3
Flask-Cors==3.0.10
gitdb==4.0.9
GitPython==3.1.27
google-api-core==2.10.1
google-auth==2.11.1
googleapis-common-protos==1.56.4
gunicorn==20.1.0
idna==3.4
importlib-metadata==4.12.0
inference-schema==1.4.2.1
itsdangerous==2.1.2
Jinja2==3.1.2
MarkupSafe==2.1.1
mlflow-skinny==1.17.0
msal==1.19.0
msal-extensions==1.0.0
numpy==1.21.6
oauthlib==3.2.1
opencensus==0.11.0
opencensus-context==0.1.3
opencensus-ext-azure==1.1.7
pandas==1.2.4
portalocker==2.5.1
protobuf==4.21.6
psutil==5.9.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
PyJWT==2.5.0
python-dateutil==2.8.2
pytz==2022.2.1
PyYAML==6.0
requests==2.28.1
rsa==4.9
six==1.16.0
smmap==5.0.0
tabulate==0.8.10
typing-extensions==4.3.0
urllib3==1.26.12
Werkzeug==2.2.2
wrapt==1.12.1
zipp==3.8.1

2022-11-02T08:47:48,992900167+00:00 | gunicorn/run | 
2022-11-02T08:47:48,994446669+00:00 | gunicorn/run | Entry script directory: /var/mlflow_resources/.
2022-11-02T08:47:48,995821470+00:00 | gunicorn/run | 
2022-11-02T08:47:48,997181272+00:00 | gunicorn/run | ###############################################
2022-11-02T08:47:48,998454473+00:00 | gunicorn/run | Dynamic Python Package Installation
2022-11-02T08:47:48,999890375+00:00 | gunicorn/run | ###############################################
2022-11-02T08:47:49,001372276+00:00 | gunicorn/run | 
2022-11-02T08:47:49,002691778+00:00 | gunicorn/run | Updating conda environment from /var/azureml-app/azureml-models/py-hands-model/1/model/conda.yaml !
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

Downloading and Extracting Packages

sqlite-3.39.4        | 789 KB    |            |   0% 
sqlite-3.39.4        | 789 KB    | ########## | 100% 
sqlite-3.39.4        | 789 KB    | ########## | 100% 

tk-8.6.12            | 3.3 MB    |            |   0% 
tk-8.6.12            | 3.3 MB    | ########## | 100% 
tk-8.6.12            | 3.3 MB    | ########## | 100% 

openssl-3.0.7        | 2.8 MB    |            |   0% 
openssl-3.0.7        | 2.8 MB    | ########## | 100% 
openssl-3.0.7        | 2.8 MB    | ########## | 100% 

libuuid-2.32.1       | 28 KB     |            |   0% 
libuuid-2.32.1       | 28 KB     | ########## | 100% 

pip-20.2.4           | 1.1 MB    |            |   0% 
pip-20.2.4           | 1.1 MB    | 1          |   1% 
pip-20.2.4           | 1.1 MB    | ####6      |  47% 
pip-20.2.4           | 1.1 MB    | ########## | 100% 
pip-20.2.4           | 1.1 MB    | ########## | 100% 

libffi-3.4.2         | 57 KB     |            |   0% 
libffi-3.4.2         | 57 KB     | ########## | 100% 

ncurses-6.3          | 1002 KB   |            |   0% 
ncurses-6.3          | 1002 KB   | ########## | 100% 
ncurses-6.3          | 1002 KB   | ########## | 100% 

python-3.8.13        | 25.2 MB   |            |   0% 
python-3.8.13        | 25.2 MB   | ##5        |  26% 
python-3.8.13        | 25.2 MB   | #######9   |  80% 
python-3.8.13        | 25.2 MB   | ########## | 100% 

xz-5.2.6             | 409 KB    |            |   0% 
xz-5.2.6             | 409 KB    | ########## | 100% 
xz-5.2.6             | 409 KB    | ########## | 100% 

_libgcc_mutex-0.1    | 3 KB      |            |   0% 
_libgcc_mutex-0.1    | 3 KB      | ########## | 100% 

bzip2-1.0.8          | 484 KB    |            |   0% 
bzip2-1.0.8          | 484 KB    | ########## | 100% 
bzip2-1.0.8          | 484 KB    | ########## | 100% 

ld_impl_linux-64-2.3 | 759 KB    |            |   0% 
ld_impl_linux-64-2.3 | 759 KB    | ########## | 100% 
ld_impl_linux-64-2.3 | 759 KB    | ########## | 100% 

libgomp-12.2.0       | 455 KB    |            |   0% 
libgomp-12.2.0       | 455 KB    | ########## | 100% 

setuptools-65.5.0    | 768 KB    |            |   0% 
setuptools-65.5.0    | 768 KB    | ########## | 100% 
setuptools-65.5.0    | 768 KB    | ########## | 100% 

_openmp_mutex-4.5    | 23 KB     |            |   0% 
_openmp_mutex-4.5    | 23 KB     | ########## | 100% 

libsqlite-3.39.4     | 803 KB    |            |   0% 
libsqlite-3.39.4     | 803 KB    | ########## | 100% 
libsqlite-3.39.4     | 803 KB    | ########## | 100% 

libnsl-2.0.0         | 31 KB     |            |   0% 
libnsl-2.0.0         | 31 KB     | ########## | 100% 

ca-certificates-2022 | 150 KB    |            |   0% 
ca-certificates-2022 | 150 KB    | ########## | 100% 

libgcc-ng-12.2.0     | 931 KB    |            |   0% 
libgcc-ng-12.2.0     | 931 KB    | ########## | 100% 
libgcc-ng-12.2.0     | 931 KB    | ########## | 100% 

readline-8.1.2       | 291 KB    |            |   0% 
readline-8.1.2       | 291 KB    | ########## | 100% 

wheel-0.37.1         | 31 KB     |            |   0% 
wheel-0.37.1         | 31 KB     | ########## | 100% 

libzlib-1.2.13       | 64 KB     |            |   0% 
libzlib-1.2.13       | 64 KB     | ########## | 100% 
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Installing pip dependencies: ...working... Ran pip subprocess with arguments:
['/opt/miniconda/envs/userenv/bin/python', '-m', 'pip', 'install', '-U', '-r', '/tmp/condaenv.u9crhh54.requirements.txt']
Pip subprocess output:
Collecting mlflow
  Downloading mlflow-1.30.0-py3-none-any.whl (17.0 MB)
Collecting cffi==1.15.1
  Downloading cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442 kB)
Collecting dill==0.3.5.1
  Downloading dill-0.3.5.1-py2.py3-none-any.whl (95 kB)
Collecting google-api-core==2.10.0
  Downloading google_api_core-2.10.0-py3-none-any.whl (115 kB)
Collecting keras==2.7.0
  Downloading keras-2.7.0-py2.py3-none-any.whl (1.3 MB)
Collecting pandas==1.3.5
  Downloading pandas-1.3.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.5 MB)
Collecting pillow==9.2.0
  Downloading Pillow-9.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
Collecting scipy==1.7.3
  Downloading scipy-1.7.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (39.3 MB)
Collecting tensorflow-gpu==2.7.4
  Downloading tensorflow_gpu-2.7.4-cp38-cp38-manylinux2010_x86_64.whl (496.0 MB)
Collecting tensorflow==2.7.4
  Downloading tensorflow-2.7.4-cp38-cp38-manylinux2010_x86_64.whl (496.0 MB)
Collecting importlib-metadata!=4.7.0,<6,>=3.7.0
  Downloading importlib_metadata-5.0.0-py3-none-any.whl (21 kB)
Collecting requests<3,>=2.17.3
  Downloading requests-2.28.1-py3-none-any.whl (62 kB)
Collecting pytz<2023
  Downloading pytz-2022.6-py2.py3-none-any.whl (498 kB)
Collecting entrypoints<1
  Downloading entrypoints-0.4-py3-none-any.whl (5.3 kB)
Collecting docker<7,>=4.0.0
  Downloading docker-6.0.0-py3-none-any.whl (147 kB)
Collecting sqlalchemy<2,>=1.4.0
  Downloading SQLAlchemy-1.4.42-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
Collecting gitpython<4,>=2.1.0
  Downloading GitPython-3.1.29-py3-none-any.whl (182 kB)
Collecting sqlparse<1,>=0.4.0
  Downloading sqlparse-0.4.3-py3-none-any.whl (42 kB)
Collecting click<9,>=7.0
  Downloading click-8.1.3-py3-none-any.whl (96 kB)
Collecting packaging<22
  Downloading packaging-21.3-py3-none-any.whl (40 kB)
Collecting databricks-cli<1,>=0.8.7
  Downloading databricks-cli-0.17.3.tar.gz (77 kB)
Collecting protobuf<5,>=3.12.0
  Downloading protobuf-4.21.9-cp37-abi3-manylinux2014_x86_64.whl (408 kB)
Collecting cloudpickle<3
  Downloading cloudpickle-2.2.0-py3-none-any.whl (25 kB)
Collecting gunicorn<21; platform_system != "Windows"
  Downloading gunicorn-20.1.0-py3-none-any.whl (79 kB)
Collecting Flask<3
  Downloading Flask-2.2.2-py3-none-any.whl (101 kB)
Collecting alembic<2
  Downloading alembic-1.8.1-py3-none-any.whl (209 kB)
Collecting numpy<2
  Downloading numpy-1.23.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
Collecting prometheus-flask-exporter<1
  Downloading prometheus_flask_exporter-0.20.3-py3-none-any.whl (18 kB)
Collecting querystring-parser<2
  Downloading querystring_parser-1.2.4-py2.py3-none-any.whl (7.9 kB)
Collecting pyyaml<7,>=5.1
  Downloading PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
Collecting googleapis-common-protos<2.0dev,>=1.56.2
  Downloading googleapis_common_protos-1.56.4-py2.py3-none-any.whl (211 kB)
Collecting google-auth<3.0dev,>=1.25.0
  Downloading google_auth-2.14.0-py2.py3-none-any.whl (175 kB)
Collecting python-dateutil>=2.7.3
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting absl-py>=0.4.0
  Downloading absl_py-1.3.0-py3-none-any.whl (124 kB)
Collecting gast<0.5.0,>=0.2.1
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting wrapt>=1.11.0
  Downloading wrapt-1.14.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (81 kB)
Collecting grpcio<2.0,>=1.24.3
  Downloading grpcio-1.50.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.7 MB)
Collecting astunparse>=1.6.0
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting flatbuffers<3.0,>=1.12
  Downloading flatbuffers-2.0.7-py2.py3-none-any.whl (26 kB)
Collecting termcolor>=1.1.0
  Downloading termcolor-2.1.0-py3-none-any.whl (5.8 kB)
Collecting opt-einsum>=2.3.2
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting tensorflow-estimator<2.8,~=2.7.0rc0
  Downloading tensorflow_estimator-2.7.0-py2.py3-none-any.whl (463 kB)
Collecting tensorflow-io-gcs-filesystem>=0.21.0
  Downloading tensorflow_io_gcs_filesystem-0.27.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.4 MB)
Collecting typing-extensions>=3.6.6
  Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting google-pasta>=0.1.1
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting libclang>=9.0.1
  Downloading libclang-14.0.6-py2.py3-none-manylinux2010_x86_64.whl (14.1 MB)
Collecting six>=1.12.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Requirement already satisfied, skipping upgrade: wheel<1.0,>=0.32.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from tensorflow-gpu==2.7.4->-r /tmp/condaenv.u9crhh54.requirements.txt (line 9)) (0.37.1)
Collecting h5py>=2.9.0
  Downloading h5py-3.7.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.5 MB)
Collecting tensorboard~=2.6
  Downloading tensorboard-2.10.1-py3-none-any.whl (5.9 MB)
Collecting keras-preprocessing>=1.1.1
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting zipp>=0.5
  Downloading zipp-3.10.0-py3-none-any.whl (6.2 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2022.9.24-py3-none-any.whl (161 kB)
Collecting charset-normalizer<3,>=2
  Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting websocket-client>=0.32.0
  Downloading websocket_client-1.4.1-py3-none-any.whl (55 kB)
Collecting greenlet!=0.4.17; python_version >= "3" and (platform_machine == "aarch64" or (platform_machine == "ppc64le" or (platform_machine == "x86_64" or (platform_machine == "amd64" or (platform_machine == "AMD64" or (platform_machine == "win32" or platform_machine == "WIN32"))))))
  Downloading greenlet-2.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (541 kB)
Collecting gitdb<5,>=4.0.1
  Downloading gitdb-4.0.9-py3-none-any.whl (63 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting pyjwt>=1.7.0
  Downloading PyJWT-2.6.0-py3-none-any.whl (20 kB)
Collecting oauthlib>=3.1.0
  Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
Collecting tabulate>=0.7.7
  Downloading tabulate-0.9.0-py3-none-any.whl (35 kB)
Requirement already satisfied, skipping upgrade: setuptools>=3.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from gunicorn<21; platform_system != "Windows"->mlflow->-r /tmp/condaenv.u9crhh54.requirements.txt (line 1)) (65.5.0)
Collecting itsdangerous>=2.0
  Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting Jinja2>=3.0
  Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting Werkzeug>=2.2.2
  Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB)
Collecting Mako
  Downloading Mako-1.2.3-py3-none-any.whl (78 kB)
Collecting importlib-resources; python_version < "3.9"
  Downloading importlib_resources-5.10.0-py3-none-any.whl (34 kB)
Collecting prometheus-client
  Downloading prometheus_client-0.15.0-py3-none-any.whl (60 kB)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4; python_version >= "3.6"
  Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting cachetools<6.0,>=2.0.0
  Downloading cachetools-5.2.0-py3-none-any.whl (9.3 kB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8
  Downloading Markdown-3.4.1-py3-none-any.whl (93 kB)
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
Collecting smmap<6,>=3.0.1
  Downloading smmap-5.0.0-py3-none-any.whl (24 kB)
Collecting MarkupSafe>=2.0
  Downloading MarkupSafe-2.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Building wheels for collected packages: databricks-cli
  Building wheel for databricks-cli (setup.py): started
  Building wheel for databricks-cli (setup.py): finished with status 'done'
  Created wheel for databricks-cli: filename=databricks_cli-0.17.3-py3-none-any.whl size=139084 sha256=59576fe5a34efc9bb21e236290b9cc60d71725062c3a6d1f162a61f96bbf7a16
  Stored in directory: /home/dockeruser/.cache/pip/wheels/58/40/7c/d021d51dac18bfd095fb6837572ad2e6f1a34d221f4b1d976b
Successfully built databricks-cli
Installing collected packages: zipp, importlib-metadata, urllib3, idna, certifi, charset-normalizer, requests, pytz, entrypoints, websocket-client, pyparsing, packaging, docker, greenlet, sqlalchemy, smmap, gitdb, gitpython, sqlparse, click, pyjwt, oauthlib, tabulate, six, databricks-cli, protobuf, python-dateutil, numpy, pandas, cloudpickle, gunicorn, itsdangerous, MarkupSafe, Jinja2, Werkzeug, Flask, Mako, importlib-resources, alembic, scipy, prometheus-client, prometheus-flask-exporter, querystring-parser, pyyaml, mlflow, pycparser, cffi, dill, googleapis-common-protos, pyasn1, pyasn1-modules, rsa, cachetools, google-auth, google-api-core, keras, pillow, absl-py, gast, wrapt, grpcio, astunparse, flatbuffers, termcolor, opt-einsum, tensorflow-estimator, tensorflow-io-gcs-filesystem, typing-extensions, google-pasta, libclang, h5py, requests-oauthlib, google-auth-oauthlib, markdown, tensorboard-plugin-wit, tensorboard-data-server, tensorboard, keras-preprocessing, tensorflow-gpu, tensorflow
Successfully installed Flask-2.2.2 Jinja2-3.1.2 Mako-1.2.3 MarkupSafe-2.1.1 Werkzeug-2.2.2 absl-py-1.3.0 alembic-1.8.1 astunparse-1.6.3 cachetools-5.2.0 certifi-2022.9.24 cffi-1.15.1 charset-normalizer-2.1.1 click-8.1.3 cloudpickle-2.2.0 databricks-cli-0.17.3 dill-0.3.5.1 docker-6.0.0 entrypoints-0.4 flatbuffers-2.0.7 gast-0.4.0 gitdb-4.0.9 gitpython-3.1.29 google-api-core-2.10.0 google-auth-2.14.0 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 googleapis-common-protos-1.56.4 greenlet-2.0.0 grpcio-1.50.0 gunicorn-20.1.0 h5py-3.7.0 idna-3.4 importlib-metadata-5.0.0 importlib-resources-5.10.0 itsdangerous-2.1.2 keras-2.7.0 keras-preprocessing-1.1.2 libclang-14.0.6 markdown-3.4.1 mlflow-1.30.0 numpy-1.23.4 oauthlib-3.2.2 opt-einsum-3.3.0 packaging-21.3 pandas-1.3.5 pillow-9.2.0 prometheus-client-0.15.0 prometheus-flask-exporter-0.20.3 protobuf-4.21.9 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pyjwt-2.6.0 pyparsing-3.0.9 python-dateutil-2.8.2 pytz-2022.6 pyyaml-6.0 querystring-parser-1.2.4 requests-2.28.1 requests-oauthlib-1.3.1 rsa-4.9 scipy-1.7.3 six-1.16.0 smmap-5.0.0 sqlalchemy-1.4.42 sqlparse-0.4.3 tabulate-0.9.0 tensorboard-2.10.1 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-2.7.4 tensorflow-estimator-2.7.0 tensorflow-gpu-2.7.4 tensorflow-io-gcs-filesystem-0.27.0 termcolor-2.1.0 typing-extensions-4.4.0 urllib3-1.26.12 websocket-client-1.4.1 wrapt-1.14.1 zipp-3.10.0

done
#
# To activate this environment, use
#
#     $ conda activate userenv
#
# To deactivate an active environment, use
#
#     $ conda deactivate



==> WARNING: A newer version of conda exists. <==
  current version: 4.11.0
  latest version: 22.9.0

Please update conda by running

    $ conda update -n base -c defaults conda


Collecting azureml-inference-server-http==0.7.6
  Downloading azureml_inference_server_http-0.7.6-py3-none-any.whl (56 kB)
Collecting inference-schema~=1.4.0
  Downloading inference_schema-1.4.2.1-py3-none-any.whl (21 kB)
Collecting flask-cors~=3.0.1
  Downloading Flask_Cors-3.0.10-py2.py3-none-any.whl (14 kB)
Collecting opencensus-ext-azure~=1.1.0
  Downloading opencensus_ext_azure-1.1.7-py2.py3-none-any.whl (42 kB)
Requirement already satisfied: gunicorn==20.1.0; platform_system != "Windows" in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from azureml-inference-server-http==0.7.6) (20.1.0)
Collecting flask<2.2.0
  Downloading Flask-2.1.3-py3-none-any.whl (95 kB)
Requirement already satisfied: python-dateutil>=2.5.3 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from inference-schema~=1.4.0->azureml-inference-server-http==0.7.6) (2.8.2)
Requirement already satisfied: pytz>=2017.2 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from inference-schema~=1.4.0->azureml-inference-server-http==0.7.6) (2022.6)
Collecting wrapt<=1.12.1,>=1.11.1
  Downloading wrapt-1.12.1.tar.gz (27 kB)
Requirement already satisfied: Six in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from flask-cors~=3.0.1->azureml-inference-server-http==0.7.6) (1.16.0)
Requirement already satisfied: requests>=2.19.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (2.28.1)
Collecting azure-identity<2.0.0,>=1.5.0
  Downloading azure_identity-1.11.0-py3-none-any.whl (136 kB)
Collecting azure-core<2.0.0,>=1.12.0
  Downloading azure_core-1.26.0-py3-none-any.whl (178 kB)
Collecting psutil>=5.6.3
  Downloading psutil-5.9.3-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (295 kB)
Collecting opencensus<1.0.0,>=0.11.0
  Downloading opencensus-0.11.0-py2.py3-none-any.whl (128 kB)
Requirement already satisfied: setuptools>=3.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from gunicorn==20.1.0; platform_system != "Windows"->azureml-inference-server-http==0.7.6) (65.5.0)
Requirement already satisfied: Jinja2>=3.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from flask<2.2.0->azureml-inference-server-http==0.7.6) (3.1.2)
Requirement already satisfied: importlib-metadata>=3.6.0; python_version < "3.10" in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from flask<2.2.0->azureml-inference-server-http==0.7.6) (5.0.0)
Requirement already satisfied: Werkzeug>=2.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from flask<2.2.0->azureml-inference-server-http==0.7.6) (2.2.2)
Requirement already satisfied: itsdangerous>=2.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from flask<2.2.0->azureml-inference-server-http==0.7.6) (2.1.2)
Requirement already satisfied: click>=8.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from flask<2.2.0->azureml-inference-server-http==0.7.6) (8.1.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from requests>=2.19.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (1.26.12)
Requirement already satisfied: idna<4,>=2.5 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from requests>=2.19.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (3.4)
Requirement already satisfied: charset-normalizer<3,>=2 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from requests>=2.19.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (2.1.1)
Requirement already satisfied: certifi>=2017.4.17 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from requests>=2.19.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (2022.9.24)
Collecting cryptography>=2.5
  Downloading cryptography-38.0.3-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.2 MB)
Collecting msal-extensions<2.0.0,>=0.3.0
  Downloading msal_extensions-1.0.0-py2.py3-none-any.whl (19 kB)
Collecting msal<2.0.0,>=1.12.0
  Downloading msal-1.20.0-py2.py3-none-any.whl (90 kB)
Requirement already satisfied: typing-extensions>=4.0.1 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from azure-core<2.0.0,>=1.12.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (4.4.0)
Collecting opencensus-context>=0.1.3
  Downloading opencensus_context-0.1.3-py2.py3-none-any.whl (5.1 kB)
Requirement already satisfied: google-api-core<3.0.0,>=1.0.0; python_version >= "3.6" in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from opencensus<1.0.0,>=0.11.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (2.10.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from Jinja2>=3.0->flask<2.2.0->azureml-inference-server-http==0.7.6) (2.1.1)
Requirement already satisfied: zipp>=0.5 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from importlib-metadata>=3.6.0; python_version < "3.10"->flask<2.2.0->azureml-inference-server-http==0.7.6) (3.10.0)
Requirement already satisfied: cffi>=1.12 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from cryptography>=2.5->azure-identity<2.0.0,>=1.5.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (1.15.1)
Collecting portalocker<3,>=1.0; python_version >= "3.5" and platform_system != "Windows"
  Downloading portalocker-2.6.0-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: PyJWT[crypto]<3,>=1.0.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from msal<2.0.0,>=1.12.0->azure-identity<2.0.0,>=1.5.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (2.6.0)
Requirement already satisfied: protobuf<5.0.0dev,>=3.20.1 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from google-api-core<3.0.0,>=1.0.0; python_version >= "3.6"->opencensus<1.0.0,>=0.11.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (4.21.9)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.56.2 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from google-api-core<3.0.0,>=1.0.0; python_version >= "3.6"->opencensus<1.0.0,>=0.11.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (1.56.4)
Requirement already satisfied: google-auth<3.0dev,>=1.25.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from google-api-core<3.0.0,>=1.0.0; python_version >= "3.6"->opencensus<1.0.0,>=0.11.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (2.14.0)
Requirement already satisfied: pycparser in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from cffi>=1.12->cryptography>=2.5->azure-identity<2.0.0,>=1.5.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (2.21)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0; python_version >= "3.6"->opencensus<1.0.0,>=0.11.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (5.2.0)
Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3.6" in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0; python_version >= "3.6"->opencensus<1.0.0,>=0.11.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (4.9)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0; python_version >= "3.6"->opencensus<1.0.0,>=0.11.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (0.2.8)
Requirement already satisfied: pyasn1>=0.1.3 in /opt/miniconda/envs/userenv/lib/python3.8/site-packages (from rsa<5,>=3.1.4; python_version >= "3.6"->google-auth<3.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0; python_version >= "3.6"->opencensus<1.0.0,>=0.11.0->opencensus-ext-azure~=1.1.0->azureml-inference-server-http==0.7.6) (0.4.8)
Building wheels for collected packages: wrapt
  Building wheel for wrapt (setup.py): started
  Building wheel for wrapt (setup.py): finished with status 'done'
  Created wheel for wrapt: filename=wrapt-1.12.1-cp38-cp38-linux_x86_64.whl size=36778 sha256=1af78109e6d786c0b24bca37f0c0617019f997cb9c3348bccd01894e5c515f1d
  Stored in directory: /home/dockeruser/.cache/pip/wheels/5f/fd/9e/b6cf5890494cb8ef0b5eaff72e5d55a70fb56316007d6dfe73
Successfully built wrapt
Installing collected packages: wrapt, inference-schema, flask, flask-cors, cryptography, portalocker, msal, msal-extensions, azure-core, azure-identity, psutil, opencensus-context, opencensus, opencensus-ext-azure, azureml-inference-server-http
  Attempting uninstall: wrapt
    Found existing installation: wrapt 1.14.1
    Uninstalling wrapt-1.14.1:
      Successfully uninstalled wrapt-1.14.1
  Attempting uninstall: flask
    Found existing installation: Flask 2.2.2
    Uninstalling Flask-2.2.2:
      Successfully uninstalled Flask-2.2.2
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

tensorflow 2.7.4 requires protobuf<3.20,>=3.9.2, but you'll have protobuf 4.21.9 which is incompatible.
tensorflow-gpu 2.7.4 requires protobuf<3.20,>=3.9.2, but you'll have protobuf 4.21.9 which is incompatible.
Successfully installed azure-core-1.26.0 azure-identity-1.11.0 azureml-inference-server-http-0.7.6 cryptography-38.0.3 flask-2.1.3 flask-cors-3.0.10 inference-schema-1.4.2.1 msal-1.20.0 msal-extensions-1.0.0 opencensus-0.11.0 opencensus-context-0.1.3 opencensus-ext-azure-1.1.7 portalocker-2.6.0 psutil-5.9.3 wrapt-1.12.1
2022-11-02T08:50:36,871553524+00:00 | gunicorn/run | 
2022-11-02T08:50:36,873075926+00:00 | gunicorn/run | ###############################################
2022-11-02T08:50:36,874677327+00:00 | gunicorn/run | Checking if the Python package azureml-inference-server-http is installed
2022-11-02T08:50:36,876095628+00:00 | gunicorn/run | ###############################################
2022-11-02T08:50:36,877502929+00:00 | gunicorn/run | 
2022-11-02T08:50:37,464965511+00:00 | gunicorn/run | 
2022-11-02T08:50:37,466645513+00:00 | gunicorn/run | ###############################################
2022-11-02T08:50:37,468141614+00:00 | gunicorn/run | AzureML Inference Server
2022-11-02T08:50:37,469586615+00:00 | gunicorn/run | ###############################################
2022-11-02T08:50:37,470929116+00:00 | gunicorn/run | 
2022-11-02T08:50:37,472555418+00:00 | gunicorn/run | Starting AzureML Inference Server HTTP.

Azure ML Inferencing HTTP server v0.7.6


Server Settings
---------------
Entry Script Name: /var/mlflow_resources/mlflow_score_script.py
Model Directory: /var/azureml-app/azureml-models/py-hands-model/1
Worker Count: 1
Worker Timeout (seconds): 300
Server Port: 31311
Application Insights Enabled: false
Application Insights Key: None
Inferencing HTTP server version: azmlinfsrv/0.7.6
CORS for the specified origins: None


Server Routes
---------------
Liveness Probe: GET   127.0.0.1:31311/
Score:          POST  127.0.0.1:31311/score

Starting gunicorn 20.1.0
Listening at: http://0.0.0.0:31311 (9)
Using worker: sync
Booting worker with pid: 833
Initializing logger
2022-11-02 08:50:38,012 | root | INFO | Starting up app insights client
logging socket not found. logging not available.
logging socket not found. logging not available.
2022-11-02 08:50:38,013 | root | INFO | Starting up app insight hooks
00000000-0000-0000-0000-000000000000,/opt/miniconda/envs/userenv/lib/python3.8/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.4
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
No signature information provided for model. If no sample information was provided with the model the deployment's swagger will not include input and output schema and typing information.For more information, please see: https://aka.ms/aml-mlflow-deploy.
2022-11-02 08:50:38.903471: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/miniconda/envs/userenv/lib:/opt/miniconda/envs/amlenv/lib:
2022-11-02 08:50:38.903503: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-11-02 08:50:39,021 | root | ERROR | Traceback (most recent call last):
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/azureml_inference_server_http/server/user_script.py", line 73, in load_script
    main_module_spec.loader.exec_module(user_module)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/var/mlflow_resources/mlflow_score_script.py", line 183, in <module>
    model = load_model(model_path)
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/mlflow/pyfunc/__init__.py", line 505, in load_model
    model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/mlflow/keras.py", line 523, in _load_pyfunc
    keras_module = importlib.import_module(f.read())
  File "/opt/miniconda/envs/userenv/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 41, in <module>
    from tensorflow.python.eager import context
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/tensorflow/python/eager/context.py", line 33, in <module>
    from tensorflow.core.framework import function_pb2
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/azureml_inference_server_http/server/aml_blueprint.py", line 130, in setup
    self.user_script.load_script(AML_APP_ROOT)
  File "/opt/miniconda/envs/userenv/lib/python3.8/site-packages/azureml_inference_server_http/server/user_script.py", line 75, in load_script
    raise UserScriptImportException(ex) from ex
azureml_inference_server_http.server.user_script.UserScriptImportException: Failed to import user script because it raised an unhandled exception

Worker exiting (pid: 833)
Shutting down: Master
Reason: Worker failed to boot.
2022-11-02T08:50:39,278044697+00:00 - gunicorn/finish 3 0
2022-11-02T08:50:39,279525698+00:00 - Exit code 3 is not normal. Killing image.