Do you have examples of Yosys JSON output? If the set is large enough something like https://github.com/ncarlier/genjson could be used to generated a merged schema.

I'm thinking of use case where you initialize the assistant "in the background"

I wonder if this could composed wel with the synchronous style proposed in https://github.com/googlesamples/assistant-sdk-python/issues/358#issuecomment-552608649, where the caller would be responsible for triggering the assistant and handle the response as the return value.

Maybe by combining it decorator approach proposed in https://github.com/googlesamples/assistant-sdk-python/issues/358#issuecomment-512249889?

from google import assistant

@assistant.on_end_of_utterance()
def end_of_query():
    print("end of query")

@assistant.on_speech_recognition_result()
def in_progress_query(result)
    print("in progress query:", result)

assistant.assist('What's the weather in Tokyo?')

I like the idea of using sounddevice to handle the audio, it also decouples audio management from the SDK.

Yep, we currently use sounddevice in audio_helpers but the idea here would be to simply accept and returns numpy array and handle the sounddevice plumbing in the calling code.

If we only use one method for all the interactions however I'm not sure of what the return type should be - maybe an AssistantResult object that contains query_text, response_text and error?

Python itself should be fine w/ a single method that returns different type, and the convention should be easy to understand: if you give a T it returns a T.

It'd also still maintain the ability to optionally subscribe to assistant events, as in some applications it's neater to have some kind of event-based interface:

Would the ability of passing raw AssistRequest/Response be enough or do you need the ability to handle event at the same time of passing one of the other type?

1. Maybe Speech Commands would provide a better start point for the hotwording model, it's an open dataset that was created using crowdsourced recording of 100k+ samples on https://aiyprojects.withgoogle.com/open_speech_recording.
2. Volume is another inconsistency example of something that could also be handled with a Device Action but that's currently managed using a dedicated field in the gRPC API https://github.com/googleapis/googleapis/blob/master/google/assistant/embedded/v1alpha2/embedded_assistant.proto#L257, if those were all exposed using Device actions it would be easier to handle those consistently in upper level layer (like the one discussed in https://github.com/googlesamples/assistant-sdk-python/issues/358).
3. Part of the complexity, comes from the fact that the API itself it pretty complicated. It provides a single generic bi-directly streaming method (AssistRequest/AssistResponse) with nested message union-like types (depending if you're handling text/audio, events). It might be worth exploring simplifying the API by provided simpler dedicated methods for handling text and event separately from the audio streaming endpoint so that every languages (not just Python) could benefit for the reduced complexity.

Something else to consider would be a drastically higher level interface, with top-level module function that allow for a REPL-like experience:

>>> from google import assistant
>>> assistant.assist('What's the weather in Tokyo?')
'Clear`
>>> with open('input.wav') as f: assistant.assist(f)
<_io.BytesIO object at 0x7f8e0575b350>
>>> import sounddevice as sd
>>> sd.play(assistant.assist(sd.rec(10)))

Other popular reverse dependencies that were mentioned before:

• https://github.com/google/aiyprojects-raspbian/tree/aiyprojects/src/aiy/assistant
• https://github.com/shivasiddharth/GassistPi

@BlackLight Thanks for the follow-up an attempt at replying to your questions below:

1. I'm not aware of any plans to reverse the deprecation of the google-assistant-library, I agree that the current situation is confusing and I hope that https://github.com/googlesamples/assistant-sdk-python/commit/6e236e964c4c2d0c74b1c15bd34f4825778cac2d and https://github.com/googlesamples/assistant-sdk-python/issues/359 will help to clarify its status.
2. (continued) for hotword, integration with open model like micro_speech would provide an open and flexible alternative. I believe a sample (as proposed in https://github.com/googlesamples/assistant-sdk-python/issues/357) would be a good fit for this repo as some developers asked about it in the past (https://github.com/googlesamples/assistant-sdk-python/issues/81 https://github.com/googlesamples/assistant-sdk-python/issues/283 https://github.com/googlesamples/assistant-sdk-python/issues/261)
3. (continue) for timers and alarms, Device actions could provide a viable alternative. As Timer is a supported smarthome trait it should be possible to be register it on the device model; I don't think there is a smarthome for Alarm yet but similar functionality could be implemented using custom device action We have samples for both of those functionality in the infamous pushtotalk.py sample w/ different traits (OnOff and Blink action), flied https://github.com/googlesamples/assistant-sdk-python/issues/378 and https://github.com/googlesamples/assistant-sdk-python/issues/379 to add more specific example.
4. let's continue discussing the higher level interface in https://github.com/googlesamples/assistant-sdk-python/issues/358
5. one of the reason pushtotalk.py is verbose is because it was meant to be a sample for the lower level google-assistant-grpc bindings. It allows for more control as you interface directly to the gRPC API and provide access to raw AssistantRequest and AssistResponse allowing custom I/O and processing of audio data. We started breaking it apart into simpler samples (like audiofileinput.py) that do a better job at showing the gRPC interface without the complexity of handling multiple conversation turn. Filed https://github.com/googlesamples/assistant-sdk-python/issues/380 to explore simplifying it further (but this is somehow orthogonal to what is discussed in 2. as I think there would still be value to providing such a low level sample like this even with an higher level interface).
6. (continue) try to refine your search by filtering project that actually depend on the library https://github.com/search?q=%27google-assistant-library%27+filename%3Arequirements.txt&type=Code, filed https://github.com/googlesamples/assistant-sdk-python/issues/381 to continue the audit further

Let me know if this stir the discussion in the right direction.

As discussed in https://github.com/googlesamples/assistant-sdk-python/issues/358#issuecomment-552471651 it would be nice to audit project depending on google-assistant-grpc and google-assistant-library.

For example:

• https://github.com/search?q=%27google-assistant-grpc%27+filename%3Arequirements.txt&type=Code
• https://github.com/search?q=%27google-assistant-library%27+filename%3Arequirements.txt&type=Code

Additional processing could allow us to cluster those project by version, forks and recent activity to more effectively communicate downstream change like https://github.com/googlesamples/assistant-sdk-python/issues/359

Now that we have https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/audiofileinput.py as a standalone sample, we could simply pushtotalk further by:

• remove file input/output functionality
• remove audio_helper and use https://python-sounddevice.readthedocs.io/ async API directly

It would be also nice to start audit project that uses the google-assistant-grpc binding directly to see if they could benefit from an higher level interface.

A few I'm aware of:

• https://github.com/google/aiyprojects-raspbian/tree/aiyprojects/src/aiy/assistant
• https://github.com/shivasiddharth/GassistPi

It would be nice to show an example of registering/handling a custom device action for alarm, to bridge the functionality gap between the google-assistant-grpc and the deprecated google-assistant-library.

It would be nice to show an example of registering/handling https://developers.google.com/assistant/smarthome/traits/timer trait device action, to bridge the functionality gap between the google-assistant-grpc and the deprecated google-assistant-library.

@Kodiologist happy to merge https://github.com/hylang/tryhy/pull/7 in the meantime and redeploy http://try-hy.appspot.com/ if that helps.

@Kodiologist do you want to keep the domain (http://try-hy.appspot.com/) if not, it might be easier just to create a new project on https://console.cloud.google.com/ and re-deploy it there.

@Kodiologist happy to transfer / ownership of the domain project (if we can manage to do so).

Curious if https://wapm.io/package/python could also be potential solution

(btw https://github.com/hylang/tryhy/pull/7, sorry for the delay)

Sorry for the late reply, I didn't get notification from this repo.

@Kodiologist can you try out #7?

Sorry for the late reply, I didn't get notification from this repo.

@Kodiologist can you try out #7?

Try it out at: https://37-dot-try-hy.appspot.com/

Fixes #6

Would it makes sense to support CMSIS SVD files as an alternative input format (instead of yaml)? This could allow you to leverage peripheral descriptions that are already authored in this format.

@acidfreako The current scope of the gRPC bindings is to expose the Google Assistant Service surface to Python, so we're unlikely to add google-assistant-library-like hotwording to the google-assistant-grpc package.

But it would be nice to document how to interface with 3rd party hotwording in the form of samples, I filed https://github.com/googlesamples/assistant-sdk-python/issues/357 for tflite micro but it would be also good to link (or have official sample) for other popular OSS hotwording package (like the snowboy one shared by @BlackLight).

@acidfreako @BlackLight I published an updated version of google-assistant-library to test PyPi that add deprecation notice to clear up the situation, see https://github.com/googlesamples/assistant-sdk-python/issues/359#issuecomment-530801818

I only tested it on x86_64 for now, once tested on Raspberry Pi 0/3/4 I will publish it to the official PyPi repo, that should clear up the situation on the library deprecation and enable further update to be made to the gRPC wrapper.

Can you paste the logs with --sandbox_debug so that we get all the compiler flags and see if $GENDIR is in there.

Did you check if the file is in blaze-bin?

@balloob thanks for working on that!

Let's work together one surfacing the issue you found in the smarthome public issue tracker

@bcoe I'm trying to make GlobalOptions accept GoogleAuth as its auth attribute.

Exposing AuthClient seemed like the natural thing to do, since GoogleAuth seems to satisfy its contact, but I'm also happy to use an union type OAuth2Client | GoogleAuth if you think that's more appropriate (and it won't require exporting anything new in the auth library).

@bcoe let's discuss this in https://github.com/googleapis/nodejs-googleapis-common/issues/175 as I already started describing the usecases I'm after in the issue comments.

@bcoe let's discuss this in https://github.com/googleapis/nodejs-googleapis-common/issues/175 as I already started describing the usecases I'm after in the issue comments.

Fixes: googleapis/nodejs-googleapis-common#175. Depends-on: googleapis/google-auth-library-nodejs/pull/803.

See googleapis/nodejs-googleapis-common#175.

Is your feature request related to a problem? Please describe.

Currently, the library allows passing a google.auth.GoogleAuth as a parameters to api bindings builder functions when using JS:

const auth = new google.auth.GoogleAuth({
  scopes: ['https://www.googleapis.com/auth/homegraph']
});
const homegraph = google.homegraph({
  version: 'v1',
  auth: auth,
});

But the same thing will actually fails when using typescript because the api builder actually expect an OAuth2Client:

t.ts:9:3 - error TS2322: Type 'GoogleAuth' is not assignable to type 'string | OAuth2Client'.
  Type 'GoogleAuth' is missing the following properties from type 'OAuth2Client': certificateCache, certificateExpiry, certificateCacheFormat, refreshTokenPromises, and 42 more.

9   auth: auth,
    ~~~~

  node_modules/googleapis-common/build/src/api.d.ts:24:5
    24     auth?: OAuth2Client | string;
           ~~~~
    The expected type comes from property 'auth' which is declared here on type 'Options'


Found 1 error.

Describe the solution you'd like

It'd be nice to be able to pass any AuthClient to the api builder as the only assumption seems to be that it implement request.

Describe alternatives you've considered

Calling .getClient() and passing to result to the api builder cool work, but this prove to be inconvenient since it's async and top-level await in not there yet.

/kind question

Description:

the :register image seems to be missing the qemu-* binaries

Steps to reproduce the issue:

~ 👺 diff -Nru <(docker pull multiarch/qemu-user-static:register && docker run --rm --privileged --entryp
oint /bin/sh multiarch/qemu-user-static:register -c find) <(docker pull multiarch/qemu-user-static && doc
ker run --rm --privileged --entrypoint /bin/sh multiarch/qemu-user-static -c find)
--- /dev/fd/63  2019-10-01 10:34:37.885057387 +0900
+++ /dev/fd/62  2019-10-01 10:34:37.885057387 +0900
@@ -1,9 +1,45 @@
-register: Pulling from multiarch/qemu-user-static
-Digest: sha256:c77eb2da3597aa370f07ef970e2e0adf155172eb9d3c40e43d97aa43eef6b0c9
-Status: Image is up to date for multiarch/qemu-user-static:register
+Using default tag: latest
+latest: Pulling from multiarch/qemu-user-static
+Digest: sha256:0d5897acf5f822cec80fe40031856d417fa3c8c8cfa5d4dc66e30c74a72a450e
+Status: Image is up to date for multiarch/qemu-user-static:latest
 .
 ./usr
 ./usr/sbin
+./usr/bin
+./usr/bin/qemu-sh4eb-static
+./usr/bin/qemu-ppc64le-static
+./usr/bin/qemu-microblaze-static
+./usr/bin/qemu-sh4-static
+./usr/bin/qemu-nios2-static
+./usr/bin/qemu-mips-static
+./usr/bin/qemu-aarch64-static
+./usr/bin/qemu-sparc64-static
+./usr/bin/qemu-s390x-static
+./usr/bin/qemu-sparc-static
+./usr/bin/qemu-xtensa-static
+./usr/bin/qemu-ppc-static
+./usr/bin/qemu-arm-static
+./usr/bin/qemu-tilegx-static
+./usr/bin/qemu-i386-static
+./usr/bin/qemu-cris-static
+./usr/bin/qemu-alpha-static
+./usr/bin/qemu-armeb-static
+./usr/bin/qemu-sparc32plus-static
+./usr/bin/qemu-mips64-static
+./usr/bin/qemu-ppc64abi32-static
+./usr/bin/qemu-mipsn32el-static
+./usr/bin/qemu-xtensaeb-static
+./usr/bin/qemu-riscv64-static
+./usr/bin/qemu-mips64el-static
+./usr/bin/qemu-hppa-static
+./usr/bin/qemu-ppc64-static
+./usr/bin/qemu-mipsn32-static
+./usr/bin/qemu-m68k-static
+./usr/bin/qemu-microblazeel-static
+./usr/bin/qemu-riscv32-static
+./usr/bin/qemu-mipsel-static
+./usr/bin/qemu-aarch64_be-static
+./usr/bin/qemu-or1k-static
 ./root
 ./dev
 ./dev/core

Describe the results you received:

./usr/bin/qemu-* files are missing in the :register image

Describe the results you expected:

./usr/bin/qemu-* files are included in the :register image

Environment:

• Container application: Docker

Output of docker version

~ 🍊 docker version
Client:
 Version:           18.09.3
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        774a1f4
 Built:             Thu Feb 28 06:34:04 2019
 OS/Arch:           linux/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          18.09.3
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       774a1f4
  Built:            Thu Feb 28 05:59:55 2019
  OS/Arch:          linux/amd64
  Experimental:     false

The message https://github.com/multiarch/qemu-user-static/pull/75 seems that they should be included:

Add all the binaries in multiarch/qemu-user-static:* images to image multiarch/qemu-user-static:register.

This will cause :register --reset -p true to fail w/ sh: write error: No such file or directory is this the expected behavior?

we would be deviating from the standard ADC protocol which is somewhat risky @hiranya911 another option could be to have firebase-tools store its credentials in the ADC format (i.e: same as gcloud),

That would have the benefit to stick to the standard and works out of the box with GOOGLE_APPLICATION_CREDENTIALS env var.

firebase-admin already lookup gcloud application default credentials out of the box: https://github.com/firebase/firebase-admin-node/blob/1a7f722d2f4a9d918a29dbf6f088f0752cb74391/src/auth/credential.ts#L45 which is great!

It would awesome if it could also look up firebase-tools credentials in a similar fashion, so that:

const admin = require('firebase-admin');

admin.initializeApp({
  credential: admin.credential.applicationDefault()
});

could just work for `firebase-tools users without requiring the user to install gcloud or download service account key to their development environment.

Note you can workaround this today by doing:

const admin = require('firebase-admin');

admin.initializeApp({
  credential: admin.credential.refreshToken({type: 'authorized_user', refresh_token: (new (require('configstore'))('firebase-tools')).get('tokens.refresh_token'), client_id: require('firebase-tools/lib/api').clientId, client_secret: require('firebase-tools/lib/api').clientSecret})
});

But it seems quite fragile if firebase-tools ever decide to change the location of the refresh token / client id / secret.

Published the upcoming release announcing the deprecation to testpypi, along with a new google-assistant-sdk that remove the library sample: https://test.pypi.org/project/google-assistant-sdk/0.6.0/ https://test.pypi.org/project/google-assistant-library/1.1.0/

We plan to publish it to regular pypi in the following days if everything looks good.

And if the manifest need to specify:

• endpoint
• request type
• schema for action parameters
• schema for results Maybe something similar to OpenAPI could be used to formalize it (and that brings interesting perspective of having discoverable workload that could be uniformally moved client or server-side).

Should the manifest have a way specify side effect / results of an action ?

My current understanding of DWA explainer is that most of them expect to act as a shortcut into some side effect that would happen in a document, if there was a way to specify a schema for the return value of an action it could also allow for implementing purely functional actions that allow other components (browser runtime, other web apps, other services) to interface pragmatically with PWA/Services.

Is that outside of the scope of DWA?

@bcoe Thanks! and sorry for the lag ;)