Skip to content

Commit c8abf00

Browse files
authored
Merge pull request #5 from d4rkstar/build_api
Build api
2 parents e535a46 + 0255ef9 commit c8abf00

27 files changed

Lines changed: 1258 additions & 125 deletions

.env.example

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,10 @@ KUBERNETES_CERT_FILENAME=./tokens/ca.crt
1212
# if not set, the image is build with local tag and will not be pushed
1313
REGISTRY=
1414
# namespace is required if REGISTRY is set
15-
NAMESPACE=
15+
NAMESPACE=
16+
17+
REGISTRY_HOST=http://127.0.0.1:5000
18+
REGISTRY_USER=opsuser
19+
REGISTRY_PASS=password
20+
21+
ADMIN_API_URL=http://127.0.0.1:5002

.github/cisetup.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,3 +18,4 @@
1818
#
1919
sudo sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/local/bin
2020
sudo apt-get -y install curl wget jq
21+
pip install uv

.github/workflows/check.yml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,3 +34,12 @@ jobs:
3434
submodules: recursive
3535
- name: License
3636
uses: apache/skywalking-eyes@main
37+
- name: Set up Python 3.12
38+
uses: actions/setup-python@v4
39+
with:
40+
python-version: 3.12
41+
- name: Setup
42+
run: bash .github/cisetup.sh
43+
- name: Unit Tests
44+
run: task utest
45+
continue-on-error: false

.licenserc.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ header:
2828
- 'LICENSE'
2929
- 'NOTICE'
3030
- 'DISCLAIMER'
31+
- 'deploy/samples/requirements.txt'
3132
- '**/*.json'
3233
- '**/*.service'
3334
- '**/*.txt'

README.md

Lines changed: 28 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -34,18 +34,23 @@ Available APIs at the moment:
3434

3535
`PATCH /system/api/v1/auth/{login}` - Update the user password patching the corresponding wsku/\<login\> entry.
3636

37+
### Build API
38+
39+
`POST /system/api/v1/build` - Perform the build of a custom image and push it to repository.
40+
41+
More informations [Here](docs/DEPLOYER.md)
42+
3743
### Info API
3844

3945
`GET /system/info` - Info endpoint
4046

41-
4247
## Developer instructions
4348

4449
You need to have access to be Apache OpenServerless admin and have access to kubernetes cluster.
4550

4651
Refer to the [Apache OpenServerless installation page](https://openserverless.apache.org/docs/installation/install/docker/):
4752

48-
Give the command `task setup-developer` and it will:
53+
Give the command `task dev:setup-developer` and it will:
4954

5055
- extract the required ca.crt and token from operator service account
5156
- copy a sample .env file
@@ -61,14 +66,22 @@ Open http://localhost:5002/system/apidocs/ to see the API documentation.
6166
Taskfile supports the following tasks:
6267

6368
```yaml
64-
* build: Build the image locally
65-
* build-and-load: Build the image and loads it to local Kind cluster
66-
* buildx: Build the docker image using buildx. Set PUSH=1 to push the image to the registry.
67-
* docker-login: Login to the docker registry. Set REGISTRY=ghcr or REGISTRY=dockerhub in .env to use the respective registry.
68-
* get-tokens: Get Service Account tokens and save them to tokens directory
69-
* image-tag: Create a new tag for the current git commit.
70-
* run: Run the admin api locally, using configuration from .env file
71-
* setup-developer: Setup developer environment
69+
* build: Build the image locally
70+
* build-and-load: Build the image and loads it to local Kind cluster
71+
* buildx: Build the docker image using buildx. Set PUSH=1 to push the image to the registry.
72+
* docker-login: Login to the docker registry. Set REGISTRY=ghcr or REGISTRY=dockerhub in .env to use the respective registry.
73+
* image-tag: Create a new tag for the current git commit.
74+
* builder:cleanjobs: Clean up old jobs
75+
* builder:delete-image: Delete an image from the registry
76+
* builder:get-image: Get an image from the registry
77+
* builder:list-catalogs: List catalogs in the registry
78+
* builder:list-images: List images in a specific catalog
79+
* builder:logs: Show logs of the last build job
80+
* builder:send: Send the build to the server
81+
* builder:updatetoml: Update the buildkitd.toml file config map
82+
* dev:get-tokens: Get Service Account tokens and save them to tokens directory
83+
* dev:run: Run the admin api locally, using configuration from .env file
84+
* dev:setup-developer: Setup developer environment
7285
```
7386

7487
## Build and push
@@ -128,4 +141,8 @@ $ git push apache 0.1.0-incubating.2507270910
128141
```
129142

130143
This will trigger the build workflow, and the process will be visible at
131-
https://github.com/apache/openserverless-admin-api/actions
144+
https://github.com/apache/openserverless-admin-api/actions
145+
146+
## Additional Documentation
147+
148+
- [Deployer](docs/DEPLOYER.md)

TODO.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
<!--
2+
~ Licensed to the Apache Software Foundation (ASF) under one
3+
~ or more contributor license agreements. See the NOTICE file
4+
~ distributed with this work for additional information
5+
~ regarding copyright ownership. The ASF licenses this file
6+
~ to you under the Apache License, Version 2.0 (the
7+
~ "License"); you may not use this file except in compliance
8+
~ with the License. You may obtain a copy of the License at
9+
~
10+
~ http://www.apache.org/licenses/LICENSE-2.0
11+
~
12+
~ Unless required by applicable law or agreed to in writing,
13+
~ software distributed under the License is distributed on an
14+
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
~ KIND, either express or implied. See the License for the
16+
~ specific language governing permissions and limitations
17+
~ under the License.
18+
~
19+
-->
20+
# TODO
21+
22+
## Tests
23+
Add integration and unit tests
24+
25+
## Various
26+
27+
- [ ] `openserverless.common.whis_user_data.py` - Add `with_` blocks for other new OpenServerless Services
28+
- [ ] `openserverless.common.whisk_user_generator` - Check if `generate_whisk_user_yaml` is complete

Taskfile.yml

Lines changed: 13 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -27,32 +27,14 @@ vars:
2727

2828
dotenv:
2929
- .env
30-
31-
tasks:
3230

33-
get-tokens:
34-
desc: "Get Service Account tokens and save them to tokens directory"
35-
silent: true
36-
cmds:
37-
- mkdir -p tokens
38-
- kubectl get secret nuvolaris-wsku-secret -o jsonpath='{.data.token}' | base64 --decode > tokens/token
39-
- kubectl get secret nuvolaris-wsku-secret -o jsonpath='{.data.ca\.crt}' | base64 --decode > tokens/ca.crt
31+
includes:
32+
builder:
33+
taskfile: TaskfileBuilder.yml
34+
dev:
35+
taskfile: TaskfileDev.yml
4036

41-
setup-developer:
42-
desc: "Setup developer environment"
43-
silent: true
44-
cmds:
45-
- task: get-tokens
46-
- |
47-
if [ ! -f .env ];
48-
then cp .env.example .env
49-
echo "Please edit .env file with your local CouchDB and Kubernetes credentials"
50-
fi
51-
- |
52-
if [ ! -d .venv ];
53-
then uv venv
54-
fi
55-
- uv pip install -r pyproject.toml 2>/dev/null
37+
tasks:
5638

5739
docker-login-ghcr: >
5840
silent: true
@@ -145,8 +127,11 @@ tasks:
145127
IMG="$BASEIMG:{{.TAG}}"
146128
kind load docker-image $IMG --name=nuvolaris
147129
148-
run:
149-
desc: |
150-
Run the admin api locally, using configuration from .env file
130+
utest:
151131
cmds:
152-
- uv run -m openserverless
132+
- |
133+
for test in openserverless/common/{{.T}}*.py
134+
do echo "*** [{{.KUBE}}] $test"
135+
uv run python3 -m doctest -o ELLIPSIS $test {{.CLI_ARGS}}
136+
done
137+
silent: true

TaskfileBuilder.yml

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
#
18+
19+
version: '3'
20+
21+
tasks:
22+
23+
send:
24+
desc: Send the build to the server
25+
vars:
26+
AUTH:
27+
sh: cat ~/.wskprops | grep "AUTH" | cut -d'=' -f2 | xargs -I {}
28+
cmds:
29+
- if test -z "{{.SOURCE}}"; then echo "SOURCE IS NOT SET" && exit 1; fi
30+
- if test -z "{{.TARGET}}"; then echo "TARGET IS NOT SET" && exit 1; fi
31+
- if test -z "{{.KIND}}"; then echo "KIND IS NOT SET" && exit 1; fi
32+
- |
33+
echo '{"source": "{{.SOURCE}}", "target": "{{.TARGET}}", "kind": "{{.KIND}}", "file": "{{.REQUIREMENTS}}" }' | \
34+
curl -X POST $ADMIN_API_URL/api/v1/build -H "Content-Type: application/json" -H "Authorization: {{.AUTH}}" -d @-
35+
- sleep 5
36+
- task: logs
37+
deps:
38+
- cleanjobs
39+
# - updatetoml
40+
silent: true
41+
42+
logs:
43+
desc: Show logs of the last build job
44+
cmds:
45+
- kubectl -n nuvolaris logs $(kubectl get jobs.batch -o name | grep "build-") -c buildkit --follow
46+
silent: false
47+
48+
cleanjobs:
49+
desc: Clean up old jobs
50+
cmds:
51+
- for I in $(kubectl get jobs -n nuvolaris | grep build | awk '{ print $1 }' | tr "\n" " "); do kubectl delete job $I; done
52+
- for I in $(kubectl get cm -n nuvolaris | grep "cm-" | awk '{ print $1 }' | tr "\n" " "); do kubectl delete cm $I; done
53+
silent: true
54+
55+
updatetoml:
56+
desc: Update the buildkitd.toml file config map
57+
cmds:
58+
- |
59+
if test $(kubectl -n nuvolaris get cm -o name | grep nuvolaris-buildkitd-conf | wc -l) -gt 0;
60+
then kubectl -n nuvolaris delete configmap nuvolaris-buildkitd-conf
61+
fi
62+
- kubectl -n nuvolaris create configmap nuvolaris-buildkitd-conf --from-file=deploy/buildkit/buildkitd.toml
63+
silent: true
64+
65+
list-catalogs:
66+
desc: List catalogs in the registry
67+
cmds:
68+
- curl -u $REGISTRY_USER:$REGISTRY_PASS $REGISTRY_HOST/v2/_catalog
69+
silent: false
70+
71+
list-images:
72+
desc: List images in a specific catalog
73+
vars:
74+
CATALOG: '{{.CATALOG}}'
75+
cmds:
76+
- if test -z "{{.CATALOG}}"; then echo "CATALOG IS NOT SET" && exit 1; fi
77+
- curl -u $REGISTRY_USER:$REGISTRY_PASS $REGISTRY_HOST/v2/{{.CATALOG}}/tags/list
78+
silent: false
79+
80+
get-image:
81+
desc: Get an image from the registry
82+
vars:
83+
IMAGE: '{{.IMAGE}}'
84+
IMAGE_NAME:
85+
sh: echo '{{.IMAGE}}' | cut -d':' -f1
86+
HASH:
87+
sh: echo '{{.IMAGE}}' | cut -d':' -f2
88+
cmds:
89+
- echo "Getting image {{.IMAGE_NAME}} with hash {{.HASH}}"
90+
- curl -u $REGISTRY_USER:$REGISTRY_PASS $REGISTRY_HOST/v2/{{.IMAGE_NAME}}/manifests/{{.HASH}}
91+
silent: false
92+
93+
delete-image:
94+
desc: Delete an image from the registry
95+
vars:
96+
IMAGE: '{{.IMAGE}}'
97+
IMAGE_NAME:
98+
sh: echo '{{.IMAGE}}' | cut -d':' -f1
99+
HASH:
100+
sh: echo '{{.IMAGE}}' | cut -d':' -f2
101+
MANIFEST_DIGEST:
102+
sh: curl --silent -u $REGISTRY_USER:$REGISTRY_PASS $REGISTRY_HOST/v2/{{.IMAGE_NAME}}/manifests/{{.HASH}} | grep -i 'Docker-Content-Digest:' | awk '{print $2}' | tr -d '\r'
103+
cmds:
104+
- echo 'Deleting image {{.IMAGE}}'
105+
- echo "Deleting manifest {{.MANIFEST_DIGEST}} for image {{.IMAGE_NAME}}"
106+
- curl -u $REGISTRY_USER:$REGISTRY_PASS -X DELETE $REGISTRY_HOST/v2/{{.IMAGE_NAME}}/manifests/{{.MANIFEST_DIGEST}}
107+
silent: false

TaskfileDev.yml

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
#
18+
19+
version: '3'
20+
21+
tasks:
22+
get-tokens:
23+
desc: "Get Service Account tokens and save them to tokens directory"
24+
silent: true
25+
cmds:
26+
- mkdir -p tokens
27+
- kubectl get secret nuvolaris-wsku-secret -o jsonpath='{.data.token}' | base64 --decode > tokens/token
28+
- kubectl get secret nuvolaris-wsku-secret -o jsonpath='{.data.ca\.crt}' | base64 --decode > tokens/ca.crt
29+
30+
setup-developer:
31+
desc: "Setup developer environment"
32+
silent: true
33+
cmds:
34+
- task: get-tokens
35+
- |
36+
if [ ! -f .env ];
37+
then cp .env.example .env
38+
echo "Please edit .env file with your local CouchDB and Kubernetes credentials"
39+
fi
40+
- |
41+
if [ ! -d .venv ];
42+
then uv venv
43+
fi
44+
- uv pip install -r pyproject.toml 2>/dev/null
45+
46+
run:
47+
desc: |
48+
Run the admin api locally, using configuration from .env file
49+
cmds:
50+
- uv run -m openserverless

deploy/buildkit/buildkitd.toml

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
#
18+
# =========================
19+
# Worker OCI (rootlesskit)
20+
# =========================
21+
[worker.oci]
22+
enabled = true
23+
rootless = true
24+
no-process-sandbox = true
25+
snapshotter = "overlayfs" # usa overlayfs se il kernel lo consente
26+
27+
[worker.containerd]
28+
enabled = false
29+
30+
# =========================
31+
# Registry HTTP insicuro
32+
# =========================
33+
[registry."nuvolaris-registry-svc:5000"]
34+
insecure = true
35+
http = true
36+
37+
# =========================
38+
# Logging
39+
# =========================
40+
[log]
41+
level = "debug"

0 commit comments

Comments
 (0)