profile
viewpoint
Tim Swast tswast Google Chicago, IL https://www.timswast.com Data Engineer in Professional Services @GoogleCloudPlatform. I also hack on personal projects, mostly related to math & pixels.

googleapis/googleapis 4325

Public interface definitions of Google APIs.

quantumlib/Cirq 3149

A python framework for creating, editing, and invoking Noisy Intermediate Scale Quantum (NISQ) circuits.

GoogleCloudPlatform/awesome-google-cloud 557

A curated list of awesome stuff for Google Cloud Platform.

GoogleCloudPlatform/ml-on-gcp 431

Machine Learning on Google Cloud Platform

googleapis/python-bigquery-pandas 285

Google BigQuery connector for pandas

googleapis/python-bigquery-sqlalchemy 207

SQLAlchemy dialect for BigQuery

GoogleCloudPlatform/professional-services-data-validator 159

Utility to compare data between homogeneous or heterogeneous environments to ensure source and target tables match

GoogleCloudPlatform/slack-samples 122

Samples for running Slack integrations on Google Cloud Platform.

pydata/pydata-google-auth 29

A package providing helpers for authenticating to Google APIs.

create barnchtswast/pandas

branch : python-db-dtypes-pandas-issue28

created branch time in 12 hours

pull request commentpandas-dev/pandas

ENH: add NDArrayBackedExtensionArray to public API

In the db-dtypes package (time and date data types at the moment), we found NDArrayBackedExtensionArray to be quite useful. Unfortunately, this is a private API at the moment. Per https://github.com/googleapis/python-db-dtypes-pandas/issues/28#issuecomment-942752795, sending a PR to make this public.

tswast

comment created time in 12 hours

push eventtswast/pandas

Tim Swast

commit sha 522b548bdd2204f2ef0f0f2db412a67bf64a254c

add whatsnew

view details

push time in 12 hours

PR opened pandas-dev/pandas

ENH: add NDArrayBackedExtensionArray to public API
  • [ ] closes #xxxx
  • [ ] tests added / passed
  • [ ] Ensure all linting tests pass, see here for how to run them
  • [ ] whatsnew entry
+2 -0

0 comment

1 changed file

pr created time in 12 hours

pull request commentibis-project/ibis

refactor(api): remove the materialize method from TableExprs

@tswast The query is still correct though isn't it?

Yes. Appears to be. I'm able to execute it, which I wasn't sure I would be able to do because of the subquery location.

Unfortunately, to remove materialize I had to remove some of ibis's overly aggressive predicate pushdown.

Makes sense. BigQuery has its own query optimizer, so I'm not all too worried about that.

The generated code produced is probably not a good thing to explicitly test, as we will need to make changes from time to time to the to compilers that result in different generated code to fix bugs or add features.

We can update the tests to actually execute the SQL. It might increase test time by quite a bit, though.

cpcloud

comment created time in 14 hours

PullRequestReviewEvent

pull request commentibis-project/ibis

refactor(api): remove the materialize method from TableExprs

Sure thing. Here's the full test output, including the ibis expression:

(ibis-bigquery-dev) ➜  ibis-bigquery git:(d0e43ef) pytest tests/system/test_client.py::test_subquery_scalar_params -vv
============================= test session starts ==============================
platform darwin -- Python 3.8.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/local/Caskroom/miniconda/base/envs/ibis-bigquery-dev/bin/python
cachedir: .pytest_cache
rootdir: /Users/swast/src/github.com/ibis-project/ibis-bigquery
plugins: cov-2.12.1, mock-3.6.1
collected 1 item

tests/system/test_client.py::test_subquery_scalar_params FAILED          [100%]

=================================== FAILURES ===================================
_________________________ test_subquery_scalar_params __________________________

alltypes = BigQueryTable[r0, name=swast-scratch.ibis_gbq_testing.functional_alltypes]
  index           int64
  Unnamed_0       i..._string_col string
  string_col      string
  timestamp_col   timestamp
  year            int64
  month           int64
project_id = 'swast-scratch', dataset_id = 'ibis_gbq_testing'

    def test_subquery_scalar_params(alltypes, project_id, dataset_id):
        t = alltypes
        param = ibis.param("timestamp").name("my_param")
        expr = (
            t[["float_col", "timestamp_col", "int_col", "string_col"]][
                lambda t: t.timestamp_col < param
            ]
            .groupby("string_col")
            .aggregate(foo=lambda t: t.float_col.sum())
            .foo.count()
        )
        result = expr.compile(params={param: "20140101"})
        expected = """\
    SELECT count(`foo`) AS `count`
    FROM (
      SELECT `string_col`, sum(`float_col`) AS `foo`
      FROM (
        SELECT `float_col`, `timestamp_col`, `int_col`, `string_col`
        FROM `{}.{}.functional_alltypes`
        WHERE `timestamp_col` < @my_param
      ) t1
      GROUP BY 1
    ) t0""".format(
            project_id, dataset_id
        )
>       assert result == expected
E       AssertionError: assert ('SELECT count(`foo`) AS `count`\n'\n 'FROM (\n'\n '  SELECT `string_col`, sum(`float_col`) AS `foo`\n'\n '  FROM (\n'\n '    SELECT `float_col`, `timestamp_col`, `int_col`, `string_col`\n'\n '    FROM `swast-scratch.ibis_gbq_testing.functional_alltypes`\n'\n '  ) t1\n'\n '  WHERE `timestamp_col` < @my_param\n'\n '  GROUP BY 1\n'\n ') t0') == ('SELECT count(`foo`) AS `count`\n'\n 'FROM (\n'\n '  SELECT `string_col`, sum(`float_col`) AS `foo`\n'\n '  FROM (\n'\n '    SELECT `float_col`, `timestamp_col`, `int_col`, `string_col`\n'\n '    FROM `swast-scratch.ibis_gbq_testing.functional_alltypes`\n'\n '    WHERE `timestamp_col` < @my_param\n'\n '  ) t1\n'\n '  GROUP BY 1\n'\n ') t0')
E           SELECT count(`foo`) AS `count`
E           FROM (
E             SELECT `string_col`, sum(`float_col`) AS `foo`
E             FROM (
E               SELECT `float_col`, `timestamp_col`, `int_col`, `string_col`
E               FROM `swast-scratch.ibis_gbq_testing.functional_alltypes`
E         -     WHERE `timestamp_col` < @my_param
E             ) t1
E         +   WHERE `timestamp_col` < @my_param
E             GROUP BY 1
E           ) t0

tests/system/test_client.py:230: AssertionError
=========================== short test summary info ============================
FAILED tests/system/test_client.py::test_subquery_scalar_params - AssertionEr...
============================== 1 failed in 42.37s =============================
cpcloud

comment created time in 16 hours

pull request commentibis-project/ibis

refactor(api): remove the materialize method from TableExprs

Actually, that failure isn't just whitespace, it moved to a different subquery.

cpcloud

comment created time in 16 hours

pull request commentibis-project/ibis

refactor(api): remove the materialize method from TableExprs

I've definitely been confused by materialize() before, so +1 on this change. Tested this PR locally against the ibis-bigquery package and didn't see any failures that would have been caused by this. That said, I'm not sure if we have test cases for the overlapping equijoin predicates Wes identified.

There do appear to be some whitespace changes since the last time I ran the test suite, but that's not such a big deal:

E           SELECT count(`foo`) AS `count`
E           FROM (
E             SELECT `string_col`, sum(`float_col`) AS `foo`
E             FROM (
E               SELECT `float_col`, `timestamp_col`, `int_col`, `string_col`
E               FROM `swast-scratch.ibis_gbq_testing.functional_alltypes`
E         -     WHERE `timestamp_col` < @my_param
E             ) t1
E         +   WHERE `timestamp_col` < @my_param
E             GROUP BY 1
E           ) t0
cpcloud

comment created time in 16 hours

Pull request review commentgoogleapis/google-cloud-go

feat(bigquery/storage/managedwriter/adapt): support schema inference for proto messages

+// Copyright 2022 Google LLC+//+// Licensed under the Apache License, Version 2.0 (the "License");+// you may not use this file except in compliance with the License.+// You may obtain a copy of the License at+//+//     https://www.apache.org/licenses/LICENSE-2.0+//+// Unless required by applicable law or agreed to in writing, software+// distributed under the License is distributed on an "AS IS" BASIS,+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+// See the License for the specific language governing permissions and+// limitations under the License.++package adapt++import (+	"fmt"++	storagepb "google.golang.org/genproto/googleapis/cloud/bigquery/storage/v1"+	"google.golang.org/protobuf/proto"+	"google.golang.org/protobuf/reflect/protoreflect"+)++var scalarTypeKindMap = map[protoreflect.Kind]storagepb.TableFieldSchema_Type{+	protoreflect.BoolKind:     storagepb.TableFieldSchema_BOOL,+	protoreflect.Int32Kind:    storagepb.TableFieldSchema_INT64,+	protoreflect.Int64Kind:    storagepb.TableFieldSchema_INT64,+	protoreflect.Sint32Kind:   storagepb.TableFieldSchema_INT64,+	protoreflect.Uint32Kind:   storagepb.TableFieldSchema_INT64,+	protoreflect.Sfixed32Kind: storagepb.TableFieldSchema_INT64,+	protoreflect.Fixed32Kind:  storagepb.TableFieldSchema_INT64,+	protoreflect.Fixed64Kind:  storagepb.TableFieldSchema_INT64,+	protoreflect.Sfixed64Kind: storagepb.TableFieldSchema_INT64,+	protoreflect.FloatKind:    storagepb.TableFieldSchema_DOUBLE,+	protoreflect.DoubleKind:   storagepb.TableFieldSchema_DOUBLE,+	protoreflect.StringKind:   storagepb.TableFieldSchema_STRING,+	protoreflect.BytesKind:    storagepb.TableFieldSchema_BYTES,++	// This should be an option+	protoreflect.EnumKind: storagepb.TableFieldSchema_INT64,+}++var bqWrapperToTypeMap = map[string]storagepb.TableFieldSchema_Type{+	".google.protobuf.BoolValue":   storagepb.TableFieldSchema_BOOL,+	".google.protobuf.BytesValue":  storagepb.TableFieldSchema_BYTES,+	".google.protobuf.Int32Value":  storagepb.TableFieldSchema_INT64,+	".google.protobuf.Int64Value":  storagepb.TableFieldSchema_INT64,+	".google.protobuf.DoubleValue": storagepb.TableFieldSchema_DOUBLE,+	".google.protobuf.StringValue": storagepb.TableFieldSchema_STRING,+}++// NameStyle controls how schema inference determines column names.+type NameStyle string

Is TEXT snake_case and JSON camelCase? If so, perhaps we could use those names instead? Or does this refer to something else?

shollyman

comment created time in 17 hours

Pull request review commentgoogleapis/google-cloud-go

feat(bigquery/storage/managedwriter/adapt): support schema inference for proto messages

+// Copyright 2022 Google LLC+//+// Licensed under the Apache License, Version 2.0 (the "License");+// you may not use this file except in compliance with the License.+// You may obtain a copy of the License at+//+//     https://www.apache.org/licenses/LICENSE-2.0+//+// Unless required by applicable law or agreed to in writing, software+// distributed under the License is distributed on an "AS IS" BASIS,+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+// See the License for the specific language governing permissions and+// limitations under the License.++package adapt++import (+	"fmt"++	storagepb "google.golang.org/genproto/googleapis/cloud/bigquery/storage/v1"+	"google.golang.org/protobuf/proto"+	"google.golang.org/protobuf/reflect/protoreflect"+)++var scalarTypeKindMap = map[protoreflect.Kind]storagepb.TableFieldSchema_Type{+	protoreflect.BoolKind:     storagepb.TableFieldSchema_BOOL,+	protoreflect.Int32Kind:    storagepb.TableFieldSchema_INT64,+	protoreflect.Int64Kind:    storagepb.TableFieldSchema_INT64,+	protoreflect.Sint32Kind:   storagepb.TableFieldSchema_INT64,+	protoreflect.Uint32Kind:   storagepb.TableFieldSchema_INT64,+	protoreflect.Sfixed32Kind: storagepb.TableFieldSchema_INT64,+	protoreflect.Fixed32Kind:  storagepb.TableFieldSchema_INT64,+	protoreflect.Fixed64Kind:  storagepb.TableFieldSchema_INT64,+	protoreflect.Sfixed64Kind: storagepb.TableFieldSchema_INT64,+	protoreflect.FloatKind:    storagepb.TableFieldSchema_DOUBLE,+	protoreflect.DoubleKind:   storagepb.TableFieldSchema_DOUBLE,+	protoreflect.StringKind:   storagepb.TableFieldSchema_STRING,+	protoreflect.BytesKind:    storagepb.TableFieldSchema_BYTES,++	// This should be an option+	protoreflect.EnumKind: storagepb.TableFieldSchema_INT64,+}++var bqWrapperToTypeMap = map[string]storagepb.TableFieldSchema_Type{+	".google.protobuf.BoolValue":   storagepb.TableFieldSchema_BOOL,+	".google.protobuf.BytesValue":  storagepb.TableFieldSchema_BYTES,+	".google.protobuf.Int32Value":  storagepb.TableFieldSchema_INT64,+	".google.protobuf.Int64Value":  storagepb.TableFieldSchema_INT64,+	".google.protobuf.DoubleValue": storagepb.TableFieldSchema_DOUBLE,+	".google.protobuf.StringValue": storagepb.TableFieldSchema_STRING,+}++// NameStyle controls how schema inference determines column names.+type NameStyle string++var (+	// DefaultNameStyle uses the default name rules.+	DefaultNameStyle NameStyle = "DEFAULT"+	// TextNameStyle uses the text proto name rules.+	TextNameStyle NameStyle = "TEXT"+	// JSONNameStyle uses the json field name rules.+	JSONNameStyle NameStyle = "JSON"+)++type schemaInferer struct {+	config struct {+		nameStyle           NameStyle+		enumAsString        bool+		relaxRequiredFields bool+		allowWrapperTypes   bool

As in nested messages? Or as in well-known types?

shollyman

comment created time in 17 hours

Pull request review commentgoogleapis/google-cloud-go

feat(bigquery/storage/managedwriter/adapt): support schema inference for proto messages

+// Copyright 2022 Google LLC+//+// Licensed under the Apache License, Version 2.0 (the "License");+// you may not use this file except in compliance with the License.+// You may obtain a copy of the License at+//+//     https://www.apache.org/licenses/LICENSE-2.0+//+// Unless required by applicable law or agreed to in writing, software+// distributed under the License is distributed on an "AS IS" BASIS,+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+// See the License for the specific language governing permissions and+// limitations under the License.++package adapt++import (+	"fmt"++	storagepb "google.golang.org/genproto/googleapis/cloud/bigquery/storage/v1"+	"google.golang.org/protobuf/proto"+	"google.golang.org/protobuf/reflect/protoreflect"+)++var scalarTypeKindMap = map[protoreflect.Kind]storagepb.TableFieldSchema_Type{+	protoreflect.BoolKind:     storagepb.TableFieldSchema_BOOL,+	protoreflect.Int32Kind:    storagepb.TableFieldSchema_INT64,+	protoreflect.Int64Kind:    storagepb.TableFieldSchema_INT64,+	protoreflect.Sint32Kind:   storagepb.TableFieldSchema_INT64,+	protoreflect.Uint32Kind:   storagepb.TableFieldSchema_INT64,+	protoreflect.Sfixed32Kind: storagepb.TableFieldSchema_INT64,+	protoreflect.Fixed32Kind:  storagepb.TableFieldSchema_INT64,+	protoreflect.Fixed64Kind:  storagepb.TableFieldSchema_INT64,+	protoreflect.Sfixed64Kind: storagepb.TableFieldSchema_INT64,+	protoreflect.FloatKind:    storagepb.TableFieldSchema_DOUBLE,+	protoreflect.DoubleKind:   storagepb.TableFieldSchema_DOUBLE,+	protoreflect.StringKind:   storagepb.TableFieldSchema_STRING,+	protoreflect.BytesKind:    storagepb.TableFieldSchema_BYTES,++	// This should be an option+	protoreflect.EnumKind: storagepb.TableFieldSchema_INT64,+}++var bqWrapperToTypeMap = map[string]storagepb.TableFieldSchema_Type{+	".google.protobuf.BoolValue":   storagepb.TableFieldSchema_BOOL,+	".google.protobuf.BytesValue":  storagepb.TableFieldSchema_BYTES,+	".google.protobuf.Int32Value":  storagepb.TableFieldSchema_INT64,+	".google.protobuf.Int64Value":  storagepb.TableFieldSchema_INT64,+	".google.protobuf.DoubleValue": storagepb.TableFieldSchema_DOUBLE,+	".google.protobuf.StringValue": storagepb.TableFieldSchema_STRING,+}++// NameStyle controls how schema inference determines column names.+type NameStyle string

I see it relates to the proto reflection method names. Still could be clearer IMO.

shollyman

comment created time in 17 hours

Pull request review commentgoogleapis/google-cloud-go

feat(bigquery/storage/managedwriter/adapt): support schema inference for proto messages

+// Copyright 2022 Google LLC+//+// Licensed under the Apache License, Version 2.0 (the "License");+// you may not use this file except in compliance with the License.+// You may obtain a copy of the License at+//+//     https://www.apache.org/licenses/LICENSE-2.0+//+// Unless required by applicable law or agreed to in writing, software+// distributed under the License is distributed on an "AS IS" BASIS,+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+// See the License for the specific language governing permissions and+// limitations under the License.++package adapt++import (+	"fmt"++	storagepb "google.golang.org/genproto/googleapis/cloud/bigquery/storage/v1"+	"google.golang.org/protobuf/proto"+	"google.golang.org/protobuf/reflect/protoreflect"+)++var scalarTypeKindMap = map[protoreflect.Kind]storagepb.TableFieldSchema_Type{+	protoreflect.BoolKind:     storagepb.TableFieldSchema_BOOL,+	protoreflect.Int32Kind:    storagepb.TableFieldSchema_INT64,+	protoreflect.Int64Kind:    storagepb.TableFieldSchema_INT64,+	protoreflect.Sint32Kind:   storagepb.TableFieldSchema_INT64,+	protoreflect.Uint32Kind:   storagepb.TableFieldSchema_INT64,+	protoreflect.Sfixed32Kind: storagepb.TableFieldSchema_INT64,+	protoreflect.Fixed32Kind:  storagepb.TableFieldSchema_INT64,+	protoreflect.Fixed64Kind:  storagepb.TableFieldSchema_INT64,+	protoreflect.Sfixed64Kind: storagepb.TableFieldSchema_INT64,+	protoreflect.FloatKind:    storagepb.TableFieldSchema_DOUBLE,+	protoreflect.DoubleKind:   storagepb.TableFieldSchema_DOUBLE,+	protoreflect.StringKind:   storagepb.TableFieldSchema_STRING,+	protoreflect.BytesKind:    storagepb.TableFieldSchema_BYTES,++	// This should be an option

+1 I could see folks wanting to map this to a string as well.

shollyman

comment created time in 17 hours

PullRequestReviewEvent
PullRequestReviewEvent

issue commentgoogleapis/python-datacatalog

Permissions error in samples testing

Pretty sure it had to be Data Catalog Admin (roles/datacatalog.admin) in order to create a taxonomy.

https://cloud.google.com/iam/docs/understanding-roles#data-catalog-roles

parthea

comment created time in 19 hours

PullRequestReviewEvent

Pull request review commentGoogleCloudPlatform/professional-services-data-validator

adding scaffolding for calc field builder in config manager

 def execute_recursive_validation(self, validation_builder, grouped_fields):                     )         elif self.config_manager.primary_keys:             validation_builder.add_config_query_groups(self.config_manager.primary_keys)++        elif self.config_manager.primary_keys and len(grouped_fields) == 0:

Possible the order of these if statements is wrong. I don't think this branch will ever be reached since elif self.config_manager.primary_keys: appears above it.

renzokuken

comment created time in a day

push eventplamut/python-bigquery

Tim Swast

commit sha 9cc3c867efb5a4e358480db700e7c9328b5dfa0a

add table of changes for pandas and Model

view details

push time in 2 days

issue commentgoogleapis/python-bigquery

Calls to `create_dataset` and `get_table` are timing out

we are wondering if there is some looming deprecation we don't know about.

There is a v3 library coming very soon, but it only affects the "models" API along with some changes to default dtypes in the pandas connector. https://github.com/googleapis/python-bigquery/issues/1021

tswast

comment created time in 2 days

issue commentgoogleapis/python-bigquery

Calls to `create_dataset` and `get_table` are timing out

@drmario-gh Retries for requests timeout errors weren't added until https://github.com/googleapis/python-bigquery/pull/896

Even though this was added in 2.25.1, I recommend version 2.27.0 or later. #896 came with a default timeout which caused problems for some long running queries.

https://github.com/googleapis/python-bigquery/blob/main/CHANGELOG.md#2270-2021-09-24

tswast

comment created time in 2 days

push eventplamut/python-bigquery

WhiteSource Renovate

commit sha 750c15d8c7449776955237d01c24be7b4b66d0b5

chore(deps): update all dependencies (#1089) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [google-api-core](https://togithub.com/googleapis/python-api-core) | `==2.3.0` -> `==2.3.2` | [![age](https://badges.renovateapi.com/packages/pypi/google-api-core/2.3.2/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/google-api-core/2.3.2/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/google-api-core/2.3.2/compatibility-slim/2.3.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/google-api-core/2.3.2/confidence-slim/2.3.0)](https://docs.renovatebot.com/merge-confidence/) | | [importlib-metadata](https://togithub.com/python/importlib_metadata) | `==4.8.2` -> `==4.8.3` | [![age](https://badges.renovateapi.com/packages/pypi/importlib-metadata/4.8.3/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/importlib-metadata/4.8.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/importlib-metadata/4.8.3/compatibility-slim/4.8.2)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/importlib-metadata/4.8.3/confidence-slim/4.8.2)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>googleapis/python-api-core</summary> ### [`v2.3.2`](https://togithub.com/googleapis/python-api-core/releases/v2.3.2) [Compare Source](https://togithub.com/googleapis/python-api-core/compare/v2.3.1...v2.3.2) ##### Bug Fixes - fix: exclude function target from retry deadline exceeded exception message ([#&#8203;318](https://www.togithub.com/googleapis/python-api-core/issues/318)) ([34ebdcc](https://togithub.com/googleapis/python-api-core/commit/34ebdcc251d4f3d7d496e8e0b78847645a06650b)) ### [`v2.3.1`](https://togithub.com/googleapis/python-api-core/compare/v2.3.0...v2.3.1) [Compare Source](https://togithub.com/googleapis/python-api-core/compare/v2.3.0...v2.3.1) </details> <details> <summary>python/importlib_metadata</summary> ### [`v4.8.3`](https://togithub.com/python/importlib_metadata/blob/HEAD/CHANGES.rst#v483) [Compare Source](https://togithub.com/python/importlib_metadata/compare/v4.8.2...v4.8.3) \====== - [#&#8203;357](https://togithub.com/python/importlib_metadata/issues/357): Fixed requirement generation from egg-info when a URL requirement is given. </details> --- ### Configuration 📅 **Schedule**: At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/python-bigquery).

view details

Anthonios Partheniou

commit sha 881c081feb248e6a2b0d7ad72667924b959f8120

chore: update .repo-metadata.json (#1093)

view details

Tim Swast

commit sha 9e0ce1f4092e04b681fae4fe8ae4170f3df00e4e

test: avoid flake if connections were already open at start (#1096) Addresses flakes such as https://source.cloud.google.com/results/invocations/bb7ce2a8-d410-45bd-af12-7c04bb8cbb25/targets/cloud-devrel%2Fclient-libraries%2Fpython%2Fgoogleapis%2Fpython-bigquery%2Fpresubmit%2Fsystem-3.10/log

view details

WhiteSource Renovate

commit sha dcb8728c12f5ab0d7809a1b6cf72755dff973772

chore(deps): update all dependencies (#1091) * chore(deps): update all dependencies * don't directly depend on cachetools * remove importlib metadata as direct dependency * remove zipp which comes from transitive dependency Co-authored-by: Tim Swast <swast@google.com>

view details

WhiteSource Renovate

commit sha 82e0a7c231e823b025f7fa5b81c4695ced0f73ba

chore(deps): update all dependencies (#1098)

view details

Hugo van Kemenade

commit sha ef1f2cbdf4e72eb8ab338ce2434ba82f8acd8515

README: Update supported Python versions (#1100) Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) Fixes #<issue_number_goes_here> 🦕 To match the Trove classifiers on PyPI: ![image](https://user-images.githubusercontent.com/1324225/148646991-cc536f1f-4389-4cd9-9995-276c9aa7245a.png) https://pypi.org/project/google-cloud-bigquery/

view details

gcf-owl-bot[bot]

commit sha 18ee0b7478a77f24c575882a30eb872d24b2eaa1

chore(samples): Add check for tests in directory (#1103) Source-Link: https://github.com/googleapis/synthtool/commit/52aef91f8d25223d9dbdb4aebd94ba8eea2101f3 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:36a95b8f494e4674dc9eee9af98961293b51b86b3649942aac800ae6c1f796d4 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

view details

Tim Swast

commit sha e23114ce362e09ac72f733a640e53a561cc9ce69

fix: remove query text from exception message, use `exception.debug_message` instead (#1105) Since query text can potentially contain sensitive information, remove it from the default exception message. This information is useful for debugging, so provide a `debug_message` attribute, which is not included in the exception representation (and thus the logs). Fixes internal issue 211616590

view details

gcf-owl-bot[bot]

commit sha 073c9d383eef4f697fbacdf31daf66ccb33d365b

build: switch to release-please for tagging (#1107)

view details

release-please[bot]

commit sha 643b333d9da207021a5726b4ad01a1b4f91f530f

chore(main): release 2.32.0 (#1104) :robot: I have created a release *beep* *boop* --- ## [2.32.0](https://github.com/googleapis/python-bigquery/compare/v2.31.0...v2.32.0) (2022-01-12) ### Features * support authorized dataset entity ([#1075](https://github.com/googleapis/python-bigquery/issues/1075)) ([c098cd0](https://github.com/googleapis/python-bigquery/commit/c098cd01c755633bfaba7193dd5c044a489a5b61)) ### Bug Fixes * remove query text from exception message, use `exception.debug_message` instead ([#1105](https://github.com/googleapis/python-bigquery/issues/1105)) ([e23114c](https://github.com/googleapis/python-bigquery/commit/e23114ce362e09ac72f733a640e53a561cc9ce69)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

view details

Tim Swast

commit sha aea8d55857607ae7c2859cfabf36e5506f57beff

chore: sync main into v3 branch

view details

WhiteSource Renovate

commit sha 144cbf4ec8f0ba0199901d2128958a68b3b6f7fc

chore(deps): update all dependencies (#1101) * chore(deps): update all dependencies * test: limit deps versions by python * remove pyproj, as it is only a transitive dependency Co-authored-by: Tim Swast <swast@google.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>

view details

gcf-owl-bot[bot]

commit sha 51aced2e4354a9997ba7a9ebf7e6c72a03e35f6b

chore(python): update release.sh to use keystore (#1111) Source-Link: https://github.com/googleapis/synthtool/commit/69fda12e2994f0b595a397e8bb6e3e9f380524eb Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:ae600f36b6bc972b368367b6f83a1d91ec2c82a4a116b383d67d547c56fe6de3 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Swast <swast@google.com>

view details

Matthew Suozzo

commit sha f54d81fc5b4bb834fe5054de369f7158fff9bb25

test: make GC check resilient to broken weakrefs (#1099) * fix: Make gc check resilient to broken weakrefs In some python deployments (I believe macOS is one), weakref objects aren't guaranteed to exist across gc collections. The proposed check pattern is also used in tensorflow [0] which ran into similar issues a few years ago. [0]: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/framework/errors_test.py#L33 * fix: add pragma for coverage * Fix whitespace

view details

gcf-owl-bot[bot]

commit sha fa8aea03879363f4d12ef0c2557047b56ae08714

chore(python): Noxfile recognizes that tests can live in a folder (#1115) Source-Link: https://github.com/googleapis/synthtool/commit/4760d8dce1351d93658cb11d02a1b7ceb23ae5d7 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f0e4b51deef56bed74d3e2359c583fc104a8d6367da3984fc5c66938db738828 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

view details

Tim Swast

commit sha 727a18d2b2d1acf17ee6cb583b11eed3c0fba0df

Merge branch 'v3' into sync-v3

view details

Tim Swast

commit sha c58ba7691cf263fd9241203e2492fe2cdd0e28e1

Merge remote-tracking branch 'upstream/main' into sync-v3

view details

Tim Swast

commit sha b67b255a8f5bb7e5bdae7c3cffc25ab2389936b4

Merge pull request #1109 from googleapis/sync-v3 chore: sync main into v3 branch

view details

Tim Swast

commit sha cc8c9c009f1f2745748660fc34954be97c532ac3

Merge branch 'v3' into iss-960

view details

push time in 2 days

PR opened googleapis/python-bigquery

feat: use `StandardSqlField` class for `Model.feature_columns` and `Model.label_columns`

Since we have a manual StandardSqlField class for other features, it makes sense to use it for Model as well.

While technically a different class and type from the generated proto classes, it should act the same and avoid some breaking changes.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • [ ] Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • [ ] Ensure the tests and linter pass
  • [ ] Code coverage does not decrease (if any source code was changed)
  • [ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

+51 -4

0 comment

2 changed files

pr created time in 2 days

create barnchgoogleapis/python-bigquery

branch : tswast-model-StandardSqlFields

created branch time in 2 days

push eventplamut/python-bigquery

Tim Swast

commit sha 760fd995cde78494f7abc5b57b81b5800b5a5f4b

start documenting model properties that have changed

view details

push time in 2 days

pull request commentgoogleapis/python-bigquery-sqlalchemy

ci(python): run lint / unit tests / docs as GH actions

Failing prerelease and compliance tests should be fixed by https://github.com/googleapis/python-bigquery-sqlalchemy/pull/401

gcf-owl-bot[bot]

comment created time in 2 days

push eventgoogleapis/python-bigquery

WhiteSource Renovate

commit sha 750c15d8c7449776955237d01c24be7b4b66d0b5

chore(deps): update all dependencies (#1089) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [google-api-core](https://togithub.com/googleapis/python-api-core) | `==2.3.0` -> `==2.3.2` | [![age](https://badges.renovateapi.com/packages/pypi/google-api-core/2.3.2/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/google-api-core/2.3.2/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/google-api-core/2.3.2/compatibility-slim/2.3.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/google-api-core/2.3.2/confidence-slim/2.3.0)](https://docs.renovatebot.com/merge-confidence/) | | [importlib-metadata](https://togithub.com/python/importlib_metadata) | `==4.8.2` -> `==4.8.3` | [![age](https://badges.renovateapi.com/packages/pypi/importlib-metadata/4.8.3/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/importlib-metadata/4.8.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/importlib-metadata/4.8.3/compatibility-slim/4.8.2)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/importlib-metadata/4.8.3/confidence-slim/4.8.2)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>googleapis/python-api-core</summary> ### [`v2.3.2`](https://togithub.com/googleapis/python-api-core/releases/v2.3.2) [Compare Source](https://togithub.com/googleapis/python-api-core/compare/v2.3.1...v2.3.2) ##### Bug Fixes - fix: exclude function target from retry deadline exceeded exception message ([#&#8203;318](https://www.togithub.com/googleapis/python-api-core/issues/318)) ([34ebdcc](https://togithub.com/googleapis/python-api-core/commit/34ebdcc251d4f3d7d496e8e0b78847645a06650b)) ### [`v2.3.1`](https://togithub.com/googleapis/python-api-core/compare/v2.3.0...v2.3.1) [Compare Source](https://togithub.com/googleapis/python-api-core/compare/v2.3.0...v2.3.1) </details> <details> <summary>python/importlib_metadata</summary> ### [`v4.8.3`](https://togithub.com/python/importlib_metadata/blob/HEAD/CHANGES.rst#v483) [Compare Source](https://togithub.com/python/importlib_metadata/compare/v4.8.2...v4.8.3) \====== - [#&#8203;357](https://togithub.com/python/importlib_metadata/issues/357): Fixed requirement generation from egg-info when a URL requirement is given. </details> --- ### Configuration 📅 **Schedule**: At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/python-bigquery).

view details

Anthonios Partheniou

commit sha 881c081feb248e6a2b0d7ad72667924b959f8120

chore: update .repo-metadata.json (#1093)

view details

Tim Swast

commit sha 9e0ce1f4092e04b681fae4fe8ae4170f3df00e4e

test: avoid flake if connections were already open at start (#1096) Addresses flakes such as https://source.cloud.google.com/results/invocations/bb7ce2a8-d410-45bd-af12-7c04bb8cbb25/targets/cloud-devrel%2Fclient-libraries%2Fpython%2Fgoogleapis%2Fpython-bigquery%2Fpresubmit%2Fsystem-3.10/log

view details

WhiteSource Renovate

commit sha dcb8728c12f5ab0d7809a1b6cf72755dff973772

chore(deps): update all dependencies (#1091) * chore(deps): update all dependencies * don't directly depend on cachetools * remove importlib metadata as direct dependency * remove zipp which comes from transitive dependency Co-authored-by: Tim Swast <swast@google.com>

view details

WhiteSource Renovate

commit sha 82e0a7c231e823b025f7fa5b81c4695ced0f73ba

chore(deps): update all dependencies (#1098)

view details

Hugo van Kemenade

commit sha ef1f2cbdf4e72eb8ab338ce2434ba82f8acd8515

README: Update supported Python versions (#1100) Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) Fixes #<issue_number_goes_here> 🦕 To match the Trove classifiers on PyPI: ![image](https://user-images.githubusercontent.com/1324225/148646991-cc536f1f-4389-4cd9-9995-276c9aa7245a.png) https://pypi.org/project/google-cloud-bigquery/

view details

gcf-owl-bot[bot]

commit sha 18ee0b7478a77f24c575882a30eb872d24b2eaa1

chore(samples): Add check for tests in directory (#1103) Source-Link: https://github.com/googleapis/synthtool/commit/52aef91f8d25223d9dbdb4aebd94ba8eea2101f3 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:36a95b8f494e4674dc9eee9af98961293b51b86b3649942aac800ae6c1f796d4 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

view details

Tim Swast

commit sha e23114ce362e09ac72f733a640e53a561cc9ce69

fix: remove query text from exception message, use `exception.debug_message` instead (#1105) Since query text can potentially contain sensitive information, remove it from the default exception message. This information is useful for debugging, so provide a `debug_message` attribute, which is not included in the exception representation (and thus the logs). Fixes internal issue 211616590

view details

gcf-owl-bot[bot]

commit sha 073c9d383eef4f697fbacdf31daf66ccb33d365b

build: switch to release-please for tagging (#1107)

view details

release-please[bot]

commit sha 643b333d9da207021a5726b4ad01a1b4f91f530f

chore(main): release 2.32.0 (#1104) :robot: I have created a release *beep* *boop* --- ## [2.32.0](https://github.com/googleapis/python-bigquery/compare/v2.31.0...v2.32.0) (2022-01-12) ### Features * support authorized dataset entity ([#1075](https://github.com/googleapis/python-bigquery/issues/1075)) ([c098cd0](https://github.com/googleapis/python-bigquery/commit/c098cd01c755633bfaba7193dd5c044a489a5b61)) ### Bug Fixes * remove query text from exception message, use `exception.debug_message` instead ([#1105](https://github.com/googleapis/python-bigquery/issues/1105)) ([e23114c](https://github.com/googleapis/python-bigquery/commit/e23114ce362e09ac72f733a640e53a561cc9ce69)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

view details

Tim Swast

commit sha aea8d55857607ae7c2859cfabf36e5506f57beff

chore: sync main into v3 branch

view details

WhiteSource Renovate

commit sha 144cbf4ec8f0ba0199901d2128958a68b3b6f7fc

chore(deps): update all dependencies (#1101) * chore(deps): update all dependencies * test: limit deps versions by python * remove pyproj, as it is only a transitive dependency Co-authored-by: Tim Swast <swast@google.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>

view details

gcf-owl-bot[bot]

commit sha 51aced2e4354a9997ba7a9ebf7e6c72a03e35f6b

chore(python): update release.sh to use keystore (#1111) Source-Link: https://github.com/googleapis/synthtool/commit/69fda12e2994f0b595a397e8bb6e3e9f380524eb Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:ae600f36b6bc972b368367b6f83a1d91ec2c82a4a116b383d67d547c56fe6de3 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Swast <swast@google.com>

view details

Matthew Suozzo

commit sha f54d81fc5b4bb834fe5054de369f7158fff9bb25

test: make GC check resilient to broken weakrefs (#1099) * fix: Make gc check resilient to broken weakrefs In some python deployments (I believe macOS is one), weakref objects aren't guaranteed to exist across gc collections. The proposed check pattern is also used in tensorflow [0] which ran into similar issues a few years ago. [0]: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/framework/errors_test.py#L33 * fix: add pragma for coverage * Fix whitespace

view details

gcf-owl-bot[bot]

commit sha fa8aea03879363f4d12ef0c2557047b56ae08714

chore(python): Noxfile recognizes that tests can live in a folder (#1115) Source-Link: https://github.com/googleapis/synthtool/commit/4760d8dce1351d93658cb11d02a1b7ceb23ae5d7 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f0e4b51deef56bed74d3e2359c583fc104a8d6367da3984fc5c66938db738828 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

view details

Tim Swast

commit sha 727a18d2b2d1acf17ee6cb583b11eed3c0fba0df

Merge branch 'v3' into sync-v3

view details

Tim Swast

commit sha c58ba7691cf263fd9241203e2492fe2cdd0e28e1

Merge remote-tracking branch 'upstream/main' into sync-v3

view details

Tim Swast

commit sha b67b255a8f5bb7e5bdae7c3cffc25ab2389936b4

Merge pull request #1109 from googleapis/sync-v3 chore: sync main into v3 branch

view details

push time in 2 days

delete branch googleapis/python-bigquery

delete branch : sync-v3

delete time in 2 days

PR merged googleapis/python-bigquery

Reviewers
chore: sync main into v3 branch api: bigquery do not merge

Towards https://github.com/googleapis/python-bigquery/issues/1021

+211 -140

3 comments

17 changed files

tswast

pr closed time in 2 days

pull request commentgoogleapis/python-bigquery

chore: sync main into v3 branch

Thanks @shollyman . All of these commits have been reviewed at least once before, so a cursory look should be sufficient. The only conflicts I had to deal with were in some samples requirements.txt files.

tswast

comment created time in 2 days

more