目次
ごあいさつ
こんばんは。
今まで「アドベントカレンダー」というものを技術ブログでしか認識したことがなく、
今年初めて本物を手に取り感動したもに倉です。
今回は、最近Cloud9はサ終したし、UIは微妙に変わったしでてんやわんやなので、
様々な変更に左右されない(だろう)CloudShellを使ってlambdaで外部モジュールを読み込めるようにしていきます。
いつも通りの超絶初心者向けです。
☆やっぱりコマンドラインが最強なんやで！！☆
前提
lambdaのlayerでrequestsをぽんとアップロードしていきます。
ランタイムは「Python 3.12」を使用します。
lambdaは先に作っときました♪
また、S3を使用する場面があるので任意のものを用意してください。
手順
Pythonのバージョンをあげる
まずはCloudShellにてPython 3.12を使えるようにしていきます！
なんか知らんかったけど、CloudShellって最初っからPython 3.9が使えるみたいです。
参考：AWS CloudShell コンピューティング環境: 仕様とソフトウェア
python -V
Python 3.9.20これをさっそく3.12にあげていく～～
注意！ 以下「いいよ」と言うまでroot or sudoで実施してください。
yum install python3.12
Last metadata expiration check: 0:06:04 ago on Wed 11 Dec 2024 01:32:21 AM UTC.
Dependencies resolved.
==========================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
==========================================================================================================================================================================================================================================================================================
Installing:
python3.12 x86_64 3.12.6-1.amzn2023.0.2 amazonlinux 28 k
Installing dependencies:
mpdecimal x86_64 2.5.1-3.amzn2023.0.3 amazonlinux 101 k
python3.12-libs x86_64 3.12.6-1.amzn2023.0.2 amazonlinux 9.2 M
python3.12-pip-wheel noarch 23.2.1-4.amzn2023.0.1 amazonlinux 1.5 M
Transaction Summary
==========================================================================================================================================================================================================================================================================================
Install 4 Packages
Total download size: 11 M
Installed size: 44 M
Is this ok [y/N]: y
Downloading Packages:
(1/4): mpdecimal-2.5.1-3.amzn2023.0.3.x86_64.rpm 2.6 MB/s | 101 kB 00:00
(2/4): python3.12-3.12.6-1.amzn2023.0.2.x86_64.rpm 65 kB/s | 28 kB 00:00
(3/4): python3.12-libs-3.12.6-1.amzn2023.0.2.x86_64.rpm 18 MB/s | 9.2 MB 00:00
(4/4): python3.12-pip-wheel-23.2.1-4.amzn2023.0.1.noarch.rpm 3.1 MB/s | 1.5 MB 00:00
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 11 MB/s | 11 MB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : python3.12-pip-wheel-23.2.1-4.amzn2023.0.1.noarch 1/4
Installing : mpdecimal-2.5.1-3.amzn2023.0.3.x86_64 2/4
Installing : python3.12-3.12.6-1.amzn2023.0.2.x86_64 3/4
Installing : python3.12-libs-3.12.6-1.amzn2023.0.2.x86_64 4/4
Running scriptlet: python3.12-libs-3.12.6-1.amzn2023.0.2.x86_64 4/4
Verifying : mpdecimal-2.5.1-3.amzn2023.0.3.x86_64 1/4
Verifying : python3.12-3.12.6-1.amzn2023.0.2.x86_64 2/4
Verifying : python3.12-libs-3.12.6-1.amzn2023.0.2.x86_64 3/4
Verifying : python3.12-pip-wheel-23.2.1-4.amzn2023.0.1.noarch 4/4
Installed:
mpdecimal-2.5.1-3.amzn2023.0.3.x86_64 python3.12-3.12.6-1.amzn2023.0.2.x86_64 python3.12-libs-3.12.6-1.amzn2023.0.2.x86_64 python3.12-pip-wheel-23.2.1-4.amzn2023.0.1.noarch
Complete!インストールできたらシンボリックリンクをちまりちまりといじって……
ll /usr/bin/python*
lrwxrwxrwx. 1 root root 16 Oct 24 13:09 /usr/bin/python -> /usr/bin/python3
lrwxrwxrwx. 1 root root 9 Dec 4 20:17 /usr/bin/python3 -> python3.9
-rwxr-xr-x. 1 root root 16168 Nov 5 18:17 /usr/bin/python3.12
-rwxr-xr-x. 1 root root 16168 Dec 4 20:17 /usr/bin/python3.9ln -sf /usr/bin/python3.12 /usr/bin/python3ll /usr/bin/python*
lrwxrwxrwx. 1 root root 16 Oct 24 13:09 /usr/bin/python -> /usr/bin/python3
lrwxrwxrwx. 1 root root 19 Dec 11 01:40 /usr/bin/python3 -> /usr/bin/python3.12
-rwxr-xr-x. 1 root root 16168 Nov 5 18:17 /usr/bin/python3.12
-rwxr-xr-x. 1 root root 16168 Dec 4 20:17 /usr/bin/python3.9「python」が巡り巡って「python3.12」につながったのでおｋ！
python -V
Python 3.12.6バージョン上がりました。
pipもいいかんじに
この状態だとpipがめちゃくちゃになっているので……
pip -V
Traceback (most recent call last):
File "/usr/bin/pip", line 7, in <module>
from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/pip", line 10, in <module>
from pip._internal.main import main
ModuleNotFoundError: No module named 'pip'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/pip", line 17, in <module>
from pip._internal import main
ModuleNotFoundError: No module named 'pip'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/pip", line 20, in <module>
from pip import main
ModuleNotFoundError: No module named 'pip'
[root@ip-10-130-62-191 ~]# pip -v
Traceback (most recent call last):
File "/usr/bin/pip", line 7, in <module>
from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/pip", line 10, in <module>
from pip._internal.main import main
ModuleNotFoundError: No module named 'pip'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/pip", line 17, in <module>
from pip._internal import main
ModuleNotFoundError: No module named 'pip'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/pip", line 20, in <module>
from pip import main
ModuleNotFoundError: No module named 'pip'こうしてアップグレード！
python3 -m ensurepip --upgrade
Looking in links: /tmp/tmpaf0fpz0h
Processing /tmp/tmpaf0fpz0h/pip-23.2.1-py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-23.2.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venvこちらもいいかんじになりました。
pip -V
pip 23.2.1 from /usr/local/lib/python3.12/site-packages/pip (python 3.12)layer用のファイルを練りねり
以下はrootからログアウト or sudoなしのcloudshell-userで実行してください！
いいよ～(^^)/~~ (約束を守るもに)
cloudshell-userじゃないとaws cliコマンドがうまく動かないので、かなり気をつけてください。
まずはrequestsインストール用のディレクトリを作成。
「python」じゃないと動かなくなるので、変えないでね！
mkdir pythonrequestsをインストール——
pip install -t python/ requests
Collecting requests
Obtaining dependency information for requests from https://files.pythonhosted.org/packages/f9/9b/335f9764261e915ed497fcdeb11df5dfd6f7bf257d4a6a2a686d80da4d54/requests-2.32.3-py3-none-any.whl.metadata
Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting charset-normalizer<4,>=2 (from requests)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/16/92/92a76dc2ff3a12e69ba94e7e05168d37d0345fa08c87e1fe24d0c2a42223/charset_normalizer-3.4.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
Using cached charset_normalizer-3.4.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests)
Obtaining dependency information for idna<4,>=2.5 from https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl.metadata
Using cached idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting urllib3<3,>=1.21.1 (from requests)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/ce/d9/5f4c13cecde62396b0d3fe530a50ccea91e7dfc1ccf0e09c228841bb5ba8/urllib3-2.2.3-py3-none-any.whl.metadata
Using cached urllib3-2.2.3-py3-none-any.whl.metadata (6.5 kB)
Collecting certifi>=2017.4.17 (from requests)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/12/90/3c9ff0512038035f59d279fddeb79f5f1eccd8859f06d6163c58798b9487/certifi-2024.8.30-py3-none-any.whl.metadata
Using cached certifi-2024.8.30-py3-none-any.whl.metadata (2.2 kB)
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Using cached certifi-2024.8.30-py3-none-any.whl (167 kB)
Using cached charset_normalizer-3.4.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (143 kB)
Using cached idna-3.10-py3-none-any.whl (70 kB)
Using cached urllib3-2.2.3-py3-none-any.whl (126 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
Successfully installed certifi-2024.8.30 charset-normalizer-3.4.0 idna-3.10 requests-2.32.3 urllib3-2.2.3
[notice] A new release of pip is available: 23.2.1 -> 24.3.1
[notice] To update, run: pip install --upgrade pipできました！
ll python
total 44
drwxr-xr-x. 2 cloudshell-user cloudshell-user 4096 Dec 11 06:38 bin
drwxr-xr-x. 3 cloudshell-user cloudshell-user 4096 Dec 11 06:38 certifi
drwxr-xr-x. 2 cloudshell-user cloudshell-user 4096 Dec 11 06:38 certifi-2024.8.30.dist-info
drwxr-xr-x. 4 cloudshell-user cloudshell-user 4096 Dec 11 06:38 charset_normalizer
drwxr-xr-x. 2 cloudshell-user cloudshell-user 4096 Dec 11 06:38 charset_normalizer-3.4.0.dist-info
drwxr-xr-x. 3 cloudshell-user cloudshell-user 4096 Dec 11 06:38 idna
drwxr-xr-x. 2 cloudshell-user cloudshell-user 4096 Dec 11 06:38 idna-3.10.dist-info
drwxr-xr-x. 3 cloudshell-user cloudshell-user 4096 Dec 11 06:38 requests
drwxr-xr-x. 2 cloudshell-user cloudshell-user 4096 Dec 11 06:38 requests-2.32.3.dist-info
drwxr-xr-x. 6 cloudshell-user cloudshell-user 4096 Dec 11 06:38 urllib3
drwxr-xr-x. 3 cloudshell-user cloudshell-user 4096 Dec 11 06:38 urllib3-2.2.3.dist-infoディレクトリを圧縮——
ここのzipファイルの名前はなんでも大丈夫です！
zip -q -r layer.zip python圧縮も完了！
ll
total 1032
-rw-r--r--. 1 cloudshell-user cloudshell-user 1049138 Dec 11 06:38 layer.zip
drwxr-xr-x. 13 cloudshell-user cloudshell-user 4096 Dec 11 06:38 pythonlayer用のファイルをS3にアップロード
アップロード！
aws s3 cp layer.zip s3://【任意のS3バケット】/layer.zip
upload: ./layer.zip to s3://【任意のS3バケット】/layer.ziplayerを作成＆適用
いよいよlayerを作成して、使えるようにしていきます！
まずはlayer作成——
aws lambda publish-layer-version --layer-name 【任意のlayer名称】 --compatible-runtimes python3.12 --content S3Bucket=【先ほど指定したS3バケット】,S3Key=layer.zip
{
"Content": {
"Location": "https://awslambda-ap-ne-1-layers.s3.ap-northeast-1.amazonaws.com/snapshots/【AWSアカウントID】/【任意のlayer名称】-XXXXXXXX-130a-4dXXXXXf-72XXXXXX5a3?versionId=C_ryA92XXXXXXXXXVp0AK7Wfc5kQ&X-XXz-Security-Token=IQoJb3JpXXXXXXXXXXXXXXXXX2F%2F%2F%2F%2F%2F%2FwEaDXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXaDcXfk5PQfEx6%2FBAiAE3QRFXXXXXXXXXXXXXX9CJuC2r3E30U%2Bhwqwr8WSMNe0irXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXF%2F8BEAQaDDkxOTk4MDkXXXXXXXXXXXXXXXXXfgo%2BN9eDZsesKp4F9OySbmnzvOZXXXXXXXXXXXXXXXXX%2BytlD2ybGGziqpABOeVCvb2M0CG4DthTEEaVX93B2X1SdimXXXXXXXXXXXXXXo3AoX42zp5BO%2BDOwyr%2BFuWiwOl2rVOjXXXXXXXXXXXuXWpp3XRGBMCclHwSXXXXXXXXXXXxxIKsHFBzYRvNawpj07B4BSt7XXXXXXXXXXXT%2F9%2BPyph%2FE3r65stpadXXXXXXXXXXXXXXXOwUdFBJajzT0XXXXXXXXXXXXXXXXXXXXXXXXxVg3tfqomlpdkxzIiskZg6nnaqRjv36heXXXXXam67ok7YeUB1DRXXXXXXXXXXXXXXXvbH0H7Q0d5yR2lYO%2Bsav8PSsdLjx8ul9jF2LPh32iPU7zlUx7fQSc2C77inR936z4SnNmHMC6hoVFvpcbbAOGjbyujEuwANM5p1fxc%2BV1nyTSLiB7HJZfurblsmBE5J45HuYGBa1ljcAGcZlKWszUm2RTkHhHEZZjkyXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXzvrSnB43DCPn%2FsBi%2Bkup3p0KDCfz%2BS6BjqxAZSoX7G762KTuC8o38SIBB8rpzXXXXXXXXXXXXXXXXXXUG5MGWypm%2FyOAGNGPCzn7C4pOt7taPpAXXXXXXXXXXXXXXXXXGptIiKkwJoS8Apyd1aVrHFkW23FESzXXXXXXXXXXXXXXXXXXXXXXXXXXxd62SfGjdha562Z6Utz82GP1RcK4DvdzuL5oHtJFs4r6xCXf1g7XXXXXXXXXXXXXXXxI42mCacYhQ%3D%3D&X-Amz-Algorithm=AWXXXXXXSHA256&X-Amz-Date=202XXXXXX3959Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Credential=ASIAXXXXXXXXXX%2F20241211%2Fap-northeast-1%2Fs3%2Faws4_request&X-Amz-Signature=22XXXXa77baa3c4d1XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX08a2ee",
"CodeSha256": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxx",
"CodeSize": 1049138
},
"LayerArn": "arn:aws:lambda:ap-northeast-1:【AWSアカウントID】:layer:【任意のlayer名称】",
"LayerVersionArn": "arn:aws:lambda:ap-northeast-1:【AWSアカウントID】:layer:【任意のlayer名称】:1",
"Description": "",
"CreatedDate": "2024-12-11T06:40:04.136+0000",
"Version": 5,
"CompatibleRuntimes": [
"python3.12"
]
}作成できたので、ARNを見に行ってみる。
aws lambda list-layer-versions --layer-name 【任意のlayer名称】
{
"LayerVersions": [
{
"LayerVersionArn": "arn:aws:lambda:ap-northeast-1:【AWSアカウントID】:layer:【任意のlayer名称】:1",
"Version": 1,
"CreatedDate": "2024-12-11T06:41:50.989+0000",
"CompatibleRuntimes": [
"python3.12"
]
}
]
}「LayerVersionArn」のところを控えましょう。
したらば、lambdaに適用！
aws lambda update-function-configuration --function-name 【lambdaの関数名】 --layers 【LayerVersionArn】
{
"FunctionName": "【lambdaの関数名】",
"FunctionArn": "arn:aws:lambda:ap-northeast-1:【AWSアカウントID】:function:【lambdaの関数名】",
"Runtime": "python3.12",
"Role": "【ロールのARN】",
"Handler": "lambda_function.lambda_handler",
"CodeSize": 372,
"Description": "",
"Timeout": 3,
"MemorySize": 128,
"LastModified": "2024-12-11T06:43:50.000+0000",
"CodeSha256": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"Version": "$LATEST",
"TracingConfig": {
"Mode": "PassThrough"
},
"RevisionId": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"Layers": [
{
"Arn": "【LayerVersionArn】",
"CodeSize": 1049138
}
],
"State": "Active",
"LastUpdateStatus": "InProgress",
"LastUpdateStatusReason": "The function is being created.",
"LastUpdateStatusReasonCode": "Creating",
"PackageType": "Zip",
"Architectures": [
"x86_64"
],
"EphemeralStorage": {
"Size": 512
},
"SnapStart": {
"ApplyOn": "None",
"OptimizationStatus": "Off"
},
"RuntimeVersionConfig": {
"RuntimeVersionArn": "arn:aws:lambda:ap-northeast-1::runtime:7515e00d6763496e7a147ffa395ef5b0f0c1ffd6064130abb5ecde5a6d630e86"
},
"LoggingConfig": {
"LogFormat": "Text",
"LogGroup": "/aws/lambda/【lambdaの関数名】"
}「Layers」のところにばっちり入ってますね。これでOK！！！
あとは好きなように使ってください！！！
あとがき
今年手にしたアドベントカレンダーは、お風呂に浮かべるあひるが出てくるやつです。
たまごのひび割れから身が見え始めたエンジニア。