profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/liamcoatman/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

tomcatling/black-nb 29

Runs black on code cells in a Jupyter notebook

liamcoatman/pandas-stfc-summer-school 5

Teaching materials for 2018 STFC summer school

liamcoatman/pandas-ucl 4

Teaching materials for Machine Learning Crash course @ UCL

facultyai/faculty-mill 3

Run and publish parameterised Jupyter notebooks using the Faculty platform

liamcoatman/nb-filter-cells 3

Filter Jupyter notebook cells by tag.

liamcoatman/HTMLTable 2

Convert a pandas dataframe in to a html table

liamcoatman/SpectraTools 1

Collection of Python routines to reduce analyse spectra

liamcoatman/FittingTools 0

Python fitting routines

startedlux-org/lux

started time in 14 hours

starteddeepmind/optax

started time in 3 days

fork davidwhogg/website

The blog for the Astronomical Data Group at the CCA

https://astrodata.nyc

fork in 4 days

startedjesseduffield/lazygit

started time in 4 days

release MartinHeinz/blog-deploy

2.6.4

released time in 5 days

release MartinHeinz/blog-deploy

2.6.3

released time in 6 days

startedkcp-dev/kcp

started time in 6 days

startedadrn/totoro

started time in 7 days

startedandymatuschak/orbit

started time in 7 days

startedmypyc/mypyc

started time in 7 days

startedfacebookincubator/cinder

started time in 7 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

  HTTP_METHODS = ["GET", "POST", "PUT", "DELETE", "PATCH"] +STREAM_RESPONSE = [+    "id: 0",+    "event: mock event",+    "data: {}",+    " ",+    "id: 1",+    "event: mock event",+    "data: {}" " ",+]

makes sense, thanks!

taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.+    session : faculty.session.Session+        The session to use to make requests.+    """+    def latest_environment_execution(self):+        """Get the latest environment execution on the server.+        +        Returns+        -------+        Execution+            The latest environment execution on a server. +        """+        return self._get("/execution/latest", _ExecutionSchema())++    def stream_server_events(self, endpoint):+        """Read the server events stream from an endpoint.+        +        Parameters+        ----------+        endpoint : str+            HTTP request endpoint.++        Yields+        ------+        ServerSentEventMessage+        """+        with self._stream(endpoint) as stream:+            for message in stream:+                yield message++    def stream_server_resources(self):+        """Stream the resources used by the server.+        +        Yields+        ------+        ServerResources+        """++        schema = _ServerResourcesSchema()+        for message in self.stream_server_events("/events"):+            if message.event == SERVER_RESOURCES_EVENT:+                yield schema.load(json.loads(message.data))++    def stream_environment_execution_step_logs(self, execution_id, step_id):+        """Read from the environment step logs.+        +        Parameters+        ----------+        execution_id : Execution.id+            ID of the environment execution.+        step_id : EnvironmentExecutionStep.id+            ID of the environment execution step.++        Yields+        ------+        EnvironmentExecutionStepLog++        """+        endpoint = "/execution/{}/executor/{}/logs".format(+            execution_id, step_id+        )+        schema = _EnvironmentExecutionStepLogSchema()+        for message in self.stream_server_events(endpoint):+            if message.event == "log":+                for line in json.loads(message.data):+                    yield schema.load(line)++    ++class _EnvironmentExecutionStepLogSchema(BaseSchema):++    line_number = fields.Integer(data_key="lineNumber", required=True)+    content = fields.String(required=True)++    @post_load+    def make_environment_execution_step_log(self, data, **kwargs):+        return EnvironmentExecutionStepLog(**data)+++class _EnvironmentExecutionStepSchema(BaseSchema):++    id = fields.UUID(required=True)+    command = fields.List(fields.String(required=True), required=True)+    status = EnumField(EnvironmentExecutionStepStatus, by_value=True, required=True)+    started_at = fields.DateTime(data_key="startedAt", missing=None)+    finished_at = fields.DateTime(data_key="finishedAt", missing=None)++    @post_load+    def make_environment_execution_step(self, data, **kwargs):+        return EnvironmentExecutionStep(**data)+++class _EnvironmentExecutionSchema(BaseSchema):++    id = fields.UUID(data_key="environmentId", required=True)+    steps = fields.List(+        fields.Nested(_EnvironmentExecutionStepSchema), required=True+    )++    @post_load+    def make_environment_execution(self, data, **kwargs):+        return EnvironmentExecution(**data)+++class _ExecutionSchema(BaseSchema):++    id = fields.UUID(data_key="executionId", required=True)+    status = EnumField(ExecutionStatus, by_value=True, required=True)+    environments = fields.List(+        fields.Nested(_EnvironmentExecutionSchema), required=True

As above:

        fields.Nested(_EnvironmentExecutionSchema)
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

 def test_delete(requests_mock, session, patch_auth):     assert response == DummyObject(foo="bar")  +def test_stream(mocker):+    response_content = mocker.Mock()+    response_content.iter_lines.return_value = STREAM_RESPONSE+    mocker.patch.object(+        BaseClient,+        "_get_raw",+        return_value=response_content,+    )++    client = BaseClient(mocker.Mock(), mocker.Mock())+    with client._stream("endpoint") as sse_stream:+        for client_sse, mock_sse in zip(sse_stream, SSE_MOCK_MESSAGES):+            assert client_sse == mock_sse

zip happily accepts sequences of different lengths, and stops iterating when the shortest sequence it is passed ends. In this case, if sse_stream returned more entries than specified by SSE_MOCK_MESSAGES, this test would not catch it. I think it would be a bit safer to instead do:

    with client._stream("endpoint") as sse_stream:
        messages = list(sse_stream)
    assert messages == SSE_MOCK_MESSAGES
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.

(also apply equivalent to lines below)

taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

  HTTP_METHODS = ["GET", "POST", "PUT", "DELETE", "PATCH"] +STREAM_RESPONSE = [+    "id: 0",+    "event: mock event",+    "data: {}",+    " ",+    "id: 1",+    "event: mock event",+    "data: {}" " ",+]

Is this supposed to be:

STREAM_RESPONSE = [
    "id: 0",
    "event: mock event",
    "data: {}",
    " ",
    "id: 1",
    "event: mock event",
    "data: {}",
    " ",
]

I think what happened here is that you wrote:

    ...
    "id: 1",
    "event: mock event",
    "data: {}"
    " ",
]

(missing the comma from the end of the "data" line)

then probably applied black, which joined the last two lines. The reason that it did this is that two string literals next to each other with no operators in between is valid Python, and they get concatenated, i.e.:

>>> a = "onetwo"
>>> b = "one" "two"
>>> c = "one" + "two"
>>> a == b
True
>>> b == c
True

Since there's no comma at the end of the "data" line, Black (correctly) interprets the next line as being the continuation of the same statement and joins them together. Happy to discuss tomorrow if it's not clear why this happened.

taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum

We can be more specific about the type returned here:

    status : ExecutionStatus
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.+    session : faculty.session.Session+        The session to use to make requests.+    """+    def latest_environment_execution(self):+        """Get the latest environment execution on the server.+        +        Returns+        -------+        Execution+            The latest environment execution on a server. +        """+        return self._get("/execution/latest", _ExecutionSchema())++    def stream_server_events(self, endpoint):+        """Read the server events stream from an endpoint.+        +        Parameters+        ----------+        endpoint : str+            HTTP request endpoint.++        Yields+        ------+        ServerSentEventMessage+        """+        with self._stream(endpoint) as stream:+            for message in stream:+                yield message++    def stream_server_resources(self):+        """Stream the resources used by the server.+        +        Yields+        ------+        ServerResources+        """++        schema = _ServerResourcesSchema()+        for message in self.stream_server_events("/events"):+            if message.event == SERVER_RESOURCES_EVENT:+                yield schema.load(json.loads(message.data))++    def stream_environment_execution_step_logs(self, execution_id, step_id):+        """Read from the environment step logs.+        +        Parameters+        ----------+        execution_id : Execution.id+            ID of the environment execution.+        step_id : EnvironmentExecutionStep.id+            ID of the environment execution step.++        Yields+        ------+        EnvironmentExecutionStepLog++        """+        endpoint = "/execution/{}/executor/{}/logs".format(+            execution_id, step_id+        )+        schema = _EnvironmentExecutionStepLogSchema()+        for message in self.stream_server_events(endpoint):+            if message.event == "log":+                for line in json.loads(message.data):+                    yield schema.load(line)++    ++class _EnvironmentExecutionStepLogSchema(BaseSchema):++    line_number = fields.Integer(data_key="lineNumber", required=True)+    content = fields.String(required=True)++    @post_load+    def make_environment_execution_step_log(self, data, **kwargs):+        return EnvironmentExecutionStepLog(**data)+++class _EnvironmentExecutionStepSchema(BaseSchema):++    id = fields.UUID(required=True)+    command = fields.List(fields.String(required=True), required=True)

required=True is redundant for entries of a list (if an entry is missing, it'll mean there's an empty list, which is valid).

    command = fields.List(fields.String(), required=True)
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.+    session : faculty.session.Session+        The session to use to make requests.+    """+    def latest_environment_execution(self):+        """Get the latest environment execution on the server.+        +        Returns+        -------+        Execution+            The latest environment execution on a server. +        """+        return self._get("/execution/latest", _ExecutionSchema())++    def stream_server_events(self, endpoint):+        """Read the server events stream from an endpoint.+        +        Parameters+        ----------+        endpoint : str+            HTTP request endpoint.++        Yields+        ------+        ServerSentEventMessage+        """+        with self._stream(endpoint) as stream:+            for message in stream:+                yield message++    def stream_server_resources(self):+        """Stream the resources used by the server.+        +        Yields+        ------+        ServerResources+        """++        schema = _ServerResourcesSchema()+        for message in self.stream_server_events("/events"):+            if message.event == SERVER_RESOURCES_EVENT:+                yield schema.load(json.loads(message.data))++    def stream_environment_execution_step_logs(self, execution_id, step_id):+        """Read from the environment step logs.+        +        Parameters+        ----------+        execution_id : Execution.id+            ID of the environment execution.+        step_id : EnvironmentExecutionStep.id+            ID of the environment execution step.++        Yields+        ------+        EnvironmentExecutionStepLog++        """+        endpoint = "/execution/{}/executor/{}/logs".format(+            execution_id, step_id+        )+        schema = _EnvironmentExecutionStepLogSchema()+        for message in self.stream_server_events(endpoint):+            if message.event == "log":+                for line in json.loads(message.data):+                    yield schema.load(line)++    ++class _EnvironmentExecutionStepLogSchema(BaseSchema):++    line_number = fields.Integer(data_key="lineNumber", required=True)+    content = fields.String(required=True)++    @post_load+    def make_environment_execution_step_log(self, data, **kwargs):+        return EnvironmentExecutionStepLog(**data)+++class _EnvironmentExecutionStepSchema(BaseSchema):++    id = fields.UUID(required=True)+    command = fields.List(fields.String(required=True), required=True)+    status = EnumField(EnvironmentExecutionStepStatus, by_value=True, required=True)+    started_at = fields.DateTime(data_key="startedAt", missing=None)+    finished_at = fields.DateTime(data_key="finishedAt", missing=None)++    @post_load+    def make_environment_execution_step(self, data, **kwargs):+        return EnvironmentExecutionStep(**data)+++class _EnvironmentExecutionSchema(BaseSchema):++    id = fields.UUID(data_key="environmentId", required=True)+    steps = fields.List(+        fields.Nested(_EnvironmentExecutionStepSchema), required=True

As above:

        fields.Nested(_EnvironmentExecutionStepSchema)
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.+    session : faculty.session.Session+        The session to use to make requests.+    """+    def latest_environment_execution(self):+        """Get the latest environment execution on the server.+        +        Returns+        -------+        Execution+            The latest environment execution on a server. +        """+        return self._get("/execution/latest", _ExecutionSchema())++    def stream_server_events(self, endpoint):+        """Read the server events stream from an endpoint.+        +        Parameters+        ----------+        endpoint : str+            HTTP request endpoint.++        Yields+        ------+        ServerSentEventMessage+        """+        with self._stream(endpoint) as stream:+            for message in stream:+                yield message++    def stream_server_resources(self):+        """Stream the resources used by the server.+        +        Yields+        ------+        ServerResources+        """++        schema = _ServerResourcesSchema()+        for message in self.stream_server_events("/events"):+            if message.event == SERVER_RESOURCES_EVENT:

Minor detail: we have this event string as a module-level constant while in stream_environment_execution_step_logs it's embedded into the method as a string literal. Shall we make this consistent by e.g. embedding as a literal here with:

            if message.event == "@SSE/SERVER_RESOURCES_UPDATED":

(and then removing the module level constant)

taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.+    session : faculty.session.Session+        The session to use to make requests.+    """+    def latest_environment_execution(self):+        """Get the latest environment execution on the server.+        +        Returns+        -------+        Execution+            The latest environment execution on a server. +        """+        return self._get("/execution/latest", _ExecutionSchema())++    def stream_server_events(self, endpoint):+        """Read the server events stream from an endpoint.+        +        Parameters+        ----------+        endpoint : str+            HTTP request endpoint.++        Yields+        ------+        ServerSentEventMessage+        """+        with self._stream(endpoint) as stream:+            for message in stream:+                yield message++    def stream_server_resources(self):+        """Stream the resources used by the server.+        +        Yields+        ------+        ServerResources+        """++        schema = _ServerResourcesSchema()+        for message in self.stream_server_events("/events"):+            if message.event == SERVER_RESOURCES_EVENT:+                yield schema.load(json.loads(message.data))++    def stream_environment_execution_step_logs(self, execution_id, step_id):+        """Read from the environment step logs.+        +        Parameters+        ----------+        execution_id : Execution.id+            ID of the environment execution.+        step_id : EnvironmentExecutionStep.id

Similarly:

        step_id : uuid.UUID
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.+    session : faculty.session.Session+        The session to use to make requests.+    """+    def latest_environment_execution(self):+        """Get the latest environment execution on the server.+        +        Returns+        -------+        Execution+            The latest environment execution on a server. +        """+        return self._get("/execution/latest", _ExecutionSchema())++    def stream_server_events(self, endpoint):+        """Read the server events stream from an endpoint.+        +        Parameters+        ----------+        endpoint : str+            HTTP request endpoint.++        Yields+        ------+        ServerSentEventMessage+        """+        with self._stream(endpoint) as stream:+            for message in stream:+                yield message++    def stream_server_resources(self):+        """Stream the resources used by the server.+        +        Yields+        ------+        ServerResources+        """++        schema = _ServerResourcesSchema()+        for message in self.stream_server_events("/events"):+            if message.event == SERVER_RESOURCES_EVENT:+                yield schema.load(json.loads(message.data))++    def stream_environment_execution_step_logs(self, execution_id, step_id):+        """Read from the environment step logs.+        +        Parameters+        ----------+        execution_id : Execution.id+            ID of the environment execution.+        step_id : EnvironmentExecutionStep.id+            ID of the environment execution step.++        Yields+        ------+        EnvironmentExecutionStepLog+

I'd remove this empty line for consistency.

taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.+    session : faculty.session.Session+        The session to use to make requests.+    """+    def latest_environment_execution(self):+        """Get the latest environment execution on the server.+        +        Returns+        -------+        Execution+            The latest environment execution on a server. +        """+        return self._get("/execution/latest", _ExecutionSchema())++    def stream_server_events(self, endpoint):+        """Read the server events stream from an endpoint.+        +        Parameters+        ----------+        endpoint : str+            HTTP request endpoint.++        Yields+        ------+        ServerSentEventMessage+        """+        with self._stream(endpoint) as stream:+            for message in stream:+                yield message++    def stream_server_resources(self):+        """Stream the resources used by the server.+        +        Yields+        ------+        ServerResources+        """++        schema = _ServerResourcesSchema()+        for message in self.stream_server_events("/events"):+            if message.event == SERVER_RESOURCES_EVENT:+                yield schema.load(json.loads(message.data))++    def stream_environment_execution_step_logs(self, execution_id, step_id):+        """Read from the environment step logs.+        +        Parameters+        ----------+        execution_id : Execution.id

This is semantically valid, but the actual type of this is:

        execution_id : uuid.UUID
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.+    session : faculty.session.Session+        The session to use to make requests.+    """+    def latest_environment_execution(self):+        """Get the latest environment execution on the server.+        +        Returns+        -------+        Execution+            The latest environment execution on a server. +        """+        return self._get("/execution/latest", _ExecutionSchema())++    def stream_server_events(self, endpoint):+        """Read the server events stream from an endpoint.+        +        Parameters+        ----------+        endpoint : str+            HTTP request endpoint.++        Yields+        ------+        ServerSentEventMessage+        """+        with self._stream(endpoint) as stream:+            for message in stream:+                yield message++    def stream_server_resources(self):+        """Stream the resources used by the server.+        +        Yields+        ------+        ServerResources+        """++        schema = _ServerResourcesSchema()+        for message in self.stream_server_events("/events"):+            if message.event == SERVER_RESOURCES_EVENT:+                yield schema.load(json.loads(message.data))

Could slightly simplify this with:

                yield schema.loads(message.data)

See https://marshmallow.readthedocs.io/en/stable/api_reference.html#marshmallow.Schema.loads

taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.+    session : faculty.session.Session+        The session to use to make requests.+    """+    def latest_environment_execution(self):+        """Get the latest environment execution on the server.+        +        Returns+        -------+        Execution+            The latest environment execution on a server. +        """+        return self._get("/execution/latest", _ExecutionSchema())++    def stream_server_events(self, endpoint):

I'm increasingly thinking it doesn't make a huge amount of sense for this method and BaseClient._stream to be separate - maybe we can discuss at our catch up tomorrow?

taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)

How about a slightly more complete example in a "Usage" section, e.g.:

    Usage
    -----
    
    This client needs to be constructed manually with a Faculty session and
    the URL of the agent of the server you wish to access:
    
    >>> import faculty
    >>> import faculty.session
    >>> from faculty.clients import ServerAgentClient
    >>> 
    >>> server_client = faculty.client("server")
    >>> server = server_client.get(...)  # TODO: put representative example arguments here
    >>> url = # TODO: complete line to extract hound URL
    >>> 
    >>> session = faculty.session.get_session()
    >>> server_agent_client = ServerAgentClient(url, session)
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.+    used : float+        current utilised memory resource.+    cache : float+        current cache memory usage.+    rss: float+        resident set size allocation. +    """++    total = attrib()+    used = attrib()+    cache = attrib()+    rss = attrib()+++class ServerAgentClient(BaseClient):+    """Client for the Faculty server events service.++    Build this client with a session directly.++    >>> client = ServerAgenClient(url, session)++    Parameters+    ----------+    url : str+        The URL of the object storage service.
        The URL of the server agent's API.
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.
        Total CPU resource, in milli CPUs.
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.
        Currently utilised CPU resource, in milli CPUs.
taraganepola

comment created time in 8 days

Pull request review commentfacultyai/faculty

Migration of faculty_cli Hound.py

+# Copyright 2018-2021 Faculty Science Limited+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+#    http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.++"""+Interact with server agent.+"""+from contextlib import contextmanager+import json+from enum import Enum++from attr import attrs, attrib+from marshmallow import fields, post_load+from marshmallow_enum import EnumField++from faculty.clients.base import BaseSchema, BaseClient++SERVER_RESOURCES_EVENT = "@SSE/SERVER_RESOURCES_UPDATED"++class ExecutionStatus(Enum):+    """The status of an environment execution."""+    NOT_STARTED = "NOT_STARTED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++class EnvironmentExecutionStepStatus(Enum):+    """The status of an environment execution step."""+    QUEUED = "QUEUED"+    CANCELLED = "CANCELLED"+    STARTED = "STARTED"+    SUCCESS = "SUCCESS"+    FAILURE = "FAILURE"++@attrs+class Execution(object):+    """Server environment execution.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the execution.+    status : enum.Enum+        The status of the execution.+    environments : List[EnvironmentExecution]+        list of EnvironmentExecution objects for each environment applied.+    started_at : Optional[datetime]+        startime of the execution.+    finished_at : Optional[datetime]+        endtime of the execution.+    """++    id = attrib()+    status = attrib()+    environments = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecution(object):+    """An environment executed on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment.+    steps : List[EnvironmentExecutionStep]+        List of EnvironmentExecutionStep objects.+    """++    id = attrib()+    steps = attrib()+++@attrs+class EnvironmentExecutionStep(object):+    """A single environment execution step on a server.+    +    Parameters+    ----------+    id : uuid.UUID+        The ID of the environment step.+    command : List[str]+        The command executed by the step.+    status : enum.Enum+        The status of the step execution.+    started_at : Optional[datetime]+        The start time of the execution step.+    finished_at : Optional[datetime]+        The finish time of the execution step.+    """++    id = attrib()+    command = attrib()+    status = attrib()+    started_at = attrib()+    finished_at = attrib()+++@attrs+class EnvironmentExecutionStepLog(object):+    """The log for an environment execution step.+    +    Parameters+    ----------+    line_number : int+        The line number within the step log.+    content : str+        The content of the step log.+    """++    line_number = attrib()+    content = attrib()++++@attrs+class ServerResources(object):+    """Information about current server resource usage.+    +    Parameters+    ----------+    milli_cpus : CpuUsage+        current CPU utilisation.+    memory_mb : MemoryUsage+        current memory utilisation. +    """++    milli_cpus = attrib()+    memory_mb = attrib()+++@attrs+class CpuUsage(object):+    """Current CPU usage on a server.+    +    Parameters+    ----------+    total : int+        total CPU resource.+    used : int+        currently utilised CPU resource.+    """++    total = attrib()+    used = attrib()+++@attrs+class MemoryUsage(object):+    """Current Memory usage on a server.+    +    Parameters+    ----------+    total : float+        total memory resource.
        Total memory resource, in megabytes.
taraganepola

comment created time in 8 days