Move minimal service into ESG source.

Author: david-woelfle

Readme.md

@@ -8,7 +8,7 @@ Energy Management Systems (EMSs), in a sense of software computing optimized ope
 
 For further information about the framework please consult the [corresponding paper available here](https://arxiv.org/abs/2402.15230). The latter provides further details about motivation and necessity of web-services for EMS applications, an extensive discussion of the technical design underlying the Energy Service Generics framework as well documentation of other relevant  aspects.
 
-## Installation
+## Usage
 
 After checking out or downloading the source code install the package with:
 
@@ -53,26 +53,22 @@ Please consider citing us if this software and/or the accompanying [paper](https
 
 ## Development Instructions
 
-Recommend way of working on the code is to automatically run the tests in the container. The provided `docker-compose.yml` file is configured accordingly. In order to start the interactive sessions just execute:
+The recommend way of working on the code is to run it in a docker container.
 
-```bash
-docker compose up -d --build && docker compose logs -f --no-log-prefix
-```
-
-This will execute pytest every time a python file has changed. Furthermore it will spin up a minimal instance  viable service at http://localhost:8800/. You can use the latter to interactively evaluate your code changes, e.g. if working on settings that influence the interactive documentation (SwaggerUI).
-
-One you are finished get rid of the containers with:
+This repo employs a [test-driven development](https://en.wikipedia.org/wiki/Test-driven_development) schema in combination with a script that automatically runs the tests on file changes. In order to start the container execute:
 
 ```bash
-docker compose down
+docker compose -f docker-compose-autotest.yml up --no-log-prefix
 ```
 
-A shortcut (without building) for executing just the tests is:
+Furthermore, a functional service is included for interactive testing. You can use the latter to evaluate your code changes, e.g. if working on settings that influence the interactive documentation (SwaggerUI). The following code will start the service:
 
 ```bash
-docker compose up --no-log-prefix energy-service-generics-devl-autotest
+docker compose -f docker-compose-interactive-test.yml up
 ```
 
+You can access the interactive API documentation on http://localhost:8800/.
+
 ## Contact
 
 Please open a GitHub issue for any inquiry that relates to the source code. Feel free to contact [David Wölfle](https://www.fzi.de/team/david-woelfle/) directly for all other inquiries.

minimal_service/pytest.ini renamed to docker-compose-autotest.yml

@@ -1,6 +1,6 @@
 # Copyright 2024 FZI Research Center for Information Technology
 #
-# Licensed under the Apache License, Version 2.0 (the "License")#
+# Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
@@ -15,8 +15,19 @@
 # SPDX-FileCopyrightText: 2024 FZI Research Center for Information Technology
 # SPDX-License-Identifier: Apache-2.0
 
-[pytest]
-
-filterwarnings =
-    # This warning is emitted by esg.test and is pointless but cannot be disabled there.
-    ignore:.+esg\.test.*:pytest.PytestAssertRewriteWarning
+services:
+  energy-service-generics-devl-autotest:
+    container_name: energy-service-generics-devl-autotest
+    build:
+      context: ./source
+    init: true # Faster shutdown.
+    entrypoint: [ "auto-pytest" ]
+    command: [ "energy_service_generics/tests/" ]
+    tty: true # Colorize output.
+    restart: unless-stopped
+    volumes:
+      - ./source:/source/energy_service_generics/
+      - ./source/esg:/usr/local/lib/python3.11/site-packages/esg
+    user: "${USER_ID:-1000}:${GROUP_ID:-1000}"
+    environment:
+      - VERSION=latest-devl

docker-compose-interactive-test.yml

@@ -0,0 +1,61 @@
+# Copyright 2024 FZI Research Center for Information Technology
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-FileCopyrightText: 2024 FZI Research Center for Information Technology
+# SPDX-License-Identifier: Apache-2.0
+
+volumes:
+  energy-service-generics-devl-service-celery-broker:
+  energy-service-generics-devl-service-celery-results:
+
+
+services:
+  energy-service-generics-devl-service-api:
+    container_name: energy-service-generics-devl-service-api
+    build:
+      context: ./source
+      target: esg-service-pandas
+    init: true # Faster shutdown.
+    entrypoint: [ "python" ]
+    command: [ "-m", "esg.service.devl_service.api" ]
+    restart: unless-stopped
+    ports:
+      - 8800:8800
+    volumes:
+      - ./source/esg:/usr/local/lib/python3.11/site-packages/esg
+      - energy-service-generics-devl-service-celery-broker:/celery/broker/
+      - energy-service-generics-devl-service-celery-results:/celery/results/
+    environment:
+      - CELERY__NAME=devl_service
+      - 'CELERY__BROKER_URL=filesystem://'
+      - 'CELERY__FS_TRANSPORT_BASE_FOLDER=/celery/'
+      - VERSION=latest-devl
+
+  energy-service-generics-devl-service-worker:
+    container_name: energy-service-generics-devl-service-worker
+    build:
+      context: ./source
+      target: esg-service-pandas
+    init: true # Faster shutdown.
+    entrypoint: [ "celery" ]
+    command: [ "--app", "esg.service.devl_service.worker", "worker", "--loglevel=INFO" ]
+    restart: unless-stopped
+    volumes:
+      - ./source/esg:/usr/local/lib/python3.11/site-packages/esg
+      - energy-service-generics-devl-service-celery-broker:/celery/broker/
+      - energy-service-generics-devl-service-celery-results:/celery/results/
+    environment:
+      - CELERY__NAME=devl_service
+      - 'CELERY__BROKER_URL=filesystem://'
+      - 'CELERY__FS_TRANSPORT_BASE_FOLDER=/celery/'

docker-compose.yml

@@ -1,2 +1,4 @@
-# Minimal Viable Service
+# Development Service
 Here some files that allow the construction of a minimal viable service. It is only intended for development of the package and allows to verify interactively how changes in the source code affect a service, which is particularly important for code segments that are hard to test, like e.g. related to the interactive documentation.
+
+Furthermore, the development service serves as a test case for the generic service tests defined in [esg/test/generic_tests.py](../../test/generic_tests.py).

minimal_service/.gitignore

@@ -0,0 +1,18 @@
+"""
+Copyright 2024 FZI Research Center for Information Technology
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+SPDX-FileCopyrightText: 2024 FZI Research Center for Information Technology
+SPDX-License-Identifier: Apache-2.0
+"""

minimal_service/celery_broker/Readme.md

@@ -15,23 +15,23 @@
 
 SPDX-FileCopyrightText: 2024 FZI Research Center for Information Technology
 SPDX-License-Identifier: Apache-2.0
-"""
 
-from packaging.version import Version
+NOTE: Need to change something here? Check if the examples services need to
+      be adjusted too!
+"""
 
 from esg.service.api import API
 
-from data_model import RequestArguments, RequestOutput
-from data_model import FittedParameters, Observations
-from data_model import FitParameterArguments
-from worker import request_task, fit_parameters_task
+from .data_model import RequestArguments, RequestOutput
+from .data_model import FittedParameters, Observations
+from .data_model import FitParameterArguments
+from .worker import request_task, fit_parameters_task
 
 api = API(
     RequestArguments=RequestArguments,
     RequestOutput=RequestOutput,
     request_task=request_task,
     title="Minimal Viable Service",
-    version=Version("0.0.1"),
     FitParameterArguments=FitParameterArguments,
     FittedParameters=FittedParameters,
     Observations=Observations,

minimal_service/celery_results/Readme.md

@@ -15,6 +15,9 @@
 
 SPDX-FileCopyrightText: 2024 FZI Research Center for Information Technology
 SPDX-License-Identifier: Apache-2.0
+
+NOTE: Need to change something here? Check if the examples services need to
+      be adjusted too!
 """
 
 from typing import List

minimal_service/Readme.md renamed to source/esg/service/devl_service/Readme.md

@@ -17,6 +17,9 @@
 
 SPDX-FileCopyrightText: 2024 FZI Research Center for Information Technology
 SPDX-License-Identifier: Apache-2.0
+
+NOTE: Need to change something here? Check if the examples services need to
+      be adjusted too!
 """
 
 import numpy as np

source/esg/service/devl_service/__init__.py

@@ -17,16 +17,19 @@
 
 SPDX-FileCopyrightText: 2024 FZI Research Center for Information Technology
 SPDX-License-Identifier: Apache-2.0
+
+NOTE: Need to change something here? Check if the examples services need to
+      be adjusted too!
 """
 
 from esg.service.worker import celery_app_from_environ
 from esg.service.worker import invoke_fit_parameters
 from esg.service.worker import invoke_handle_request
 
-from data_model import RequestArguments, RequestOutput
-from data_model import FittedParameters, Observations
-from data_model import FitParameterArguments
-from fooc import fit_parameters, handle_request
+from .data_model import RequestArguments, RequestOutput
+from .data_model import FittedParameters, Observations
+from .data_model import FitParameterArguments
+from .fooc import fit_parameters, handle_request
 
 app = celery_app_from_environ()
 

minimal_service/api.py renamed to source/esg/service/devl_service/api.py

@@ -34,6 +34,9 @@ class GenericMessageSerializationTest:
     A generic set of tests to verify that the data can be serialized between
     the expected representations.
 
+    NOTE: This class is used in `tests/models/*.py`.
+          Check if the tests there work as expected if working on this code.
+
     Attributes:
     -----------
     ModelClass : pydantic model class
@@ -136,6 +139,9 @@ class GenericMessageSerializationTestBEMcom(GenericMessageSerializationTest):
     Extends `GenericMessageSerializationTest` with tests about serialization
     from and to BEMCom message format.
 
+    NOTE: This class is used in `tests/models/*.py`.
+          Check if the tests there work as expected if working on this code.
+
     Attributes:
     -----------
     ModelClass : pydantic model class
@@ -220,6 +226,9 @@ class GenericWorkerTaskTest(TestClassWithFixtures):
     """
     Tests for checking that the worker tasks are correctly implemented.
 
+    NOTE: This class is used in `tests/service/devl_service/test_worker.py`.
+          Check if the tests there work as expected if working on this code.
+
     Attributes:
     -----------
     fixture_names : list of str
@@ -292,6 +301,9 @@ class GenericFOOCTest(TestClassWithFixtures):
           want to add more sophisticated tests for your forecasting or
           optimization code.
 
+    NOTE: This class is used in `tests/service/devl_service/test_fooc.py`.
+          Check if the tests there work as expected if working on this code.
+
     Attributes:
     -----------
     fixture_names : list of str
@@ -362,6 +374,8 @@ class GenericAPITest:
     This basically just verifies that the models are wired up correctly as
     validating the tasks too would require a full integration with the worker.
 
+    NOTE: This class is used in `tests/service/devl_service/test_api.py`.
+          Check if the tests there work as expected if working on this code.
 
     Attributes:
     -----------