Security advisory database for Rust crates published through crates.io
Mirror of Apache Aurora
Mirror of Apache Aurora
Mirror of Apache Aurora
[WIP] Bazel remote caching and execution service
A temporary home for pants static binaries and scripts
Temporary, dumb way to host binaries at a known URL.
Source code for our Discord bot
Audit Cargo.lock files for crates with security vulnerabilities
issue commentpantsbuild/pex
Strange issue with et-xmlfile dependency
Thanks a lot @jsirois will test it as soon as the release is out
comment created time in an hour
PR opened pantsbuild/pants
This environment variable was apparently previously used to control some kind of linting that pants no longer does, so we can get rid of the code that sets it.
pr created time in 4 hours
pull request commentpantsbuild/pants
And again another suites has failed.
comment created time in 16 hours
Pull request review commentpantsbuild/pants
WIP Allow running pants with Python 3.9
def all_entries(cls) -> List[Dict[str, str]]: class PythonVersion(Enum): py37 = "py37" py38 = "py38"+ py39 = "py39" def __str__(self) -> str: return str(self.value) @property def number(self) -> int:- return {self.py37: 37, self.py38: 38}[self] # type: ignore[index]+ return {self.py37: 37, self.py38: 38, self.py39: 39}[self] # type: ignore[index] @property def decimal(self) -> float:- return {self.py37: 3.7, self.py38: 3.8}[self] # type: ignore[index]+ return {self.py37: 3.7, self.py38: 3.8, self.py39: 3.9}[self] # type: ignore[index] def default_stage(self, *, is_bootstrap: bool = False) -> Stage: if is_bootstrap:- return {self.py37: Stage.bootstrap, self.py38: Stage.bootstrap_cron}[self] # type: ignore[index]- return {self.py37: Stage.test, self.py38: Stage.test_cron}[self] # type: ignore[index]+ return {self.py37: Stage.bootstrap, self.py38: Stage.bootstrap_cron, self.py39: Stage.bootstrap_cron}[self] # type: ignore[index]+ return {self.py37: Stage.test, self.py38: Stage.test_cron, self.py39: Stage.test_cron}[self] # type: ignore[index]
Exactly, but only for the sake of testing. Before merging, we want to revert it back to what you have currently
comment created time in 20 hours
Pull request review commentpantsbuild/pants
WIP Allow running pants with Python 3.9
def all_entries(cls) -> List[Dict[str, str]]: class PythonVersion(Enum): py37 = "py37" py38 = "py38"+ py39 = "py39" def __str__(self) -> str: return str(self.value) @property def number(self) -> int:- return {self.py37: 37, self.py38: 38}[self] # type: ignore[index]+ return {self.py37: 37, self.py38: 38, self.py39: 39}[self] # type: ignore[index] @property def decimal(self) -> float:- return {self.py37: 3.7, self.py38: 3.8}[self] # type: ignore[index]+ return {self.py37: 3.7, self.py38: 3.8, self.py39: 3.9}[self] # type: ignore[index] def default_stage(self, *, is_bootstrap: bool = False) -> Stage: if is_bootstrap:- return {self.py37: Stage.bootstrap, self.py38: Stage.bootstrap_cron}[self] # type: ignore[index]- return {self.py37: Stage.test, self.py38: Stage.test_cron}[self] # type: ignore[index]+ return {self.py37: Stage.bootstrap, self.py38: Stage.bootstrap_cron, self.py39: Stage.bootstrap_cron}[self] # type: ignore[index]+ return {self.py37: Stage.test, self.py38: Stage.test_cron, self.py39: Stage.test_cron}[self] # type: ignore[index]
I'm not super-familiar with our travis configuration setup. Are you saying that this line should be:
return {self.py37: Stage.test_cron, self.py38: Stage.test_cron, self.py39: Stage.test}[self] # type: ignore[index]
with the effect of running Python 3.9 by default in ordinary builds, and running Python 3.8/Python 3.7 during nightly "cron" builds?
comment created time in 20 hours
Pull request review commentpantsbuild/pants
python_requirements: provide global module_mapping with defaults.
+---+attrs:+ - attr++enum34:+ - enum++paho_mqtt:+ - paho++pyopenssl:+ - OpenSSL++pycrypto:+ - Crypto++pymongo:+ - bson+ - gridfs++pytest_runner:+ - ptr++python_dateutil:+ - dateutil++setuptools:+ - easy_install+ - pkg_resources++zope.event:
@Eric-Arellano thoughts?
comment created time in 20 hours
Pull request review commentpantsbuild/pants
python_requirements: provide global module_mapping with defaults.
+---+attrs:+ - attr++enum34:+ - enum++paho_mqtt:+ - paho++pyopenssl:+ - OpenSSL++pycrypto:+ - Crypto++pymongo:+ - bson+ - gridfs++pytest_runner:+ - ptr++python_dateutil:+ - dateutil++setuptools:+ - easy_install+ - pkg_resources++zope.event:
I think we'd want to map to the highest package that is unique to the distribution. E.g., zope.event just maps to zope.event and not to zope. The assumption being that if you consume zope.event you are always importing from zope.event and not from zope.
That seems like the most sane assumption (and means that we can omit zope here, assuming our "dist-name == top-level package name" heuristic works with dots in the name.)
comment created time in 20 hours
pull request commentpantsbuild/pants
python_requirements: provide global module_mapping with defaults.
@benjyw what do you think about this backwards compatibility discussion? I'm leaning towards it being acceptable.
If we decide this is categorically fine, @kaos I'll leave some more detailed comments for how I suggest factoring this / where to make the change.
Hmm, yes, I think on balance that I'd rather inconvenience the probably-tiny number of people who the collision might affect (and those people have already dealt with this and are savvy) than the large number of first-time users who shouldn't have to deal with this. So I think the backwards compat risk, as you describe it, is small enough to proceed.
comment created time in 20 hours
push eventpantsbuild/pants
commit sha eed7d2e77e040044a9fb98374fec221f02042995
Simplify pants initialization process and remove `pants_exe.py` (#11643) ### Problem The code implementing pants initialization is somewhat overcomplicated and more verbose than it needs to be. ### Solution This commit: * Moves the definition of some `PANTS_` environment variables relevant to initialization to a common file, to avoid duplicating environment variable string names in different parts of the codebase * Renames `PANTS_ENTRYPOINT` to `PANTS_DAEMON_ENTRYPOINT`, to make it clearer what role this entrypoint mechanism is playing in pants execution * Folds the fairly-minimal functionality of `pants_exe.py` into `pants_loader.py` or integration test files, and removes `pants_exe.py` entirely ### Result This is a refactoring commit that should have no user-visible change
push time in 21 hours
PR merged pantsbuild/pants
Problem
The code implementing pants initialization is somewhat overcomplicated and more verbose than it needs to be.
Solution
This commit:
- Moves the definition of some
PANTS_environment variables relevant to initialization to a common file, to avoid duplicating environment variable string names in different parts of the codebase - Renames
PANTS_ENTRYPOINTtoPANTS_DAEMON_ENTRYPOINT, to make it clearer what role this entrypoint mechanism is playing in pants execution - Folds the fairly-minimal functionality of
pants_exe.pyintopants_loader.pyor integration test files, and removespants_exe.pyentirely
Result
This is a refactoring commit that should have no user-visible change
pr closed time in 21 hours
Pull request review commentpantsbuild/pants
Simplify pants initialization process and remove `pants_exe.py`
+# Copyright 2021 Pants project contributors (see CONTRIBUTORS.md).+# Licensed under the Apache License, Version 2.0 (see LICENSE).+++IGNORE_UNRECOGNIZED_ENCODING = "PANTS_IGNORE_UNRECOGNIZED_ENCODING"+RECURSION_LIMIT = "PANTS_RECURSION_LIMIT"+DAEMON_ENTRYPOINT = "PANTS_DAEMON_ENTRYPOINT"+PANTSC_PROFILE = "PANTSC_PROFILE"
Huh, I wonder why it's pantsc and not pantsd.
Is it supposed to represent "pants c_profile" perhaps?
comment created time in 21 hours
pull request commentpantsbuild/pants
Bump Zinc version to support Scala 2.13
@stuhood thank you!
I've taken a look at the CI results and I'm not sure how to interpret that. I have a gut feeling that they are false negative. There are some Yarn-related issues.
comment created time in 21 hours
Pull request review commentpantsbuild/pants
WIP Allow running pants with Python 3.9
backend_packages.add = [ "pants.backend.python.typecheck.mypy", "pants.backend.python.mixed_interpreter_constraints", "internal_plugins.releases",- "toolchain.pants.auth",- "toolchain.pants.buildsense",- "toolchain.pants.common",+# "toolchain.pants.auth",+# "toolchain.pants.buildsense",+# "toolchain.pants.common",
Yeah this is blocked by https://github.com/pantsbuild/pants/issues/11469, which Liam is working on. It's okay to temporarily turn these off.
What we will really want to do is find a way to turn off only in the Py39 jobs, but that can wait till everything is green.
comment created time in 21 hours
Pull request review commentpantsbuild/pants
WIP Allow running pants with Python 3.9
def all_entries(cls) -> List[Dict[str, str]]: class PythonVersion(Enum): py37 = "py37" py38 = "py38"+ py39 = "py39" def __str__(self) -> str: return str(self.value) @property def number(self) -> int:- return {self.py37: 37, self.py38: 38}[self] # type: ignore[index]+ return {self.py37: 37, self.py38: 38, self.py39: 39}[self] # type: ignore[index] @property def decimal(self) -> float:- return {self.py37: 3.7, self.py38: 3.8}[self] # type: ignore[index]+ return {self.py37: 3.7, self.py38: 3.8, self.py39: 3.9}[self] # type: ignore[index] def default_stage(self, *, is_bootstrap: bool = False) -> Stage: if is_bootstrap:- return {self.py37: Stage.bootstrap, self.py38: Stage.bootstrap_cron}[self] # type: ignore[index]- return {self.py37: Stage.test, self.py38: Stage.test_cron}[self] # type: ignore[index]+ return {self.py37: Stage.bootstrap, self.py38: Stage.bootstrap_cron, self.py39: Stage.bootstrap_cron}[self] # type: ignore[index]+ return {self.py37: Stage.test, self.py38: Stage.test_cron, self.py39: Stage.test_cron}[self] # type: ignore[index]
When testing, you will want to invert these so that PY39 runs during daily and the others during nightly. Although, last time I checked Py39 shards are already working and the only thing left is the Build Wheels shards, which should happen no matter what during daily.
comment created time in 21 hours
Pull request review commentpantsbuild/pants
Simplify pants initialization process and remove `pants_exe.py`
+# Copyright 2021 Pants project contributors (see CONTRIBUTORS.md).
Better to put under pants/bin as these env vars are solely useful for the bin files.
comment created time in 21 hours
Pull request review commentpantsbuild/pants
Simplify pants initialization process and remove `pants_exe.py`
+# Copyright 2021 Pants project contributors (see CONTRIBUTORS.md).+# Licensed under the Apache License, Version 2.0 (see LICENSE).+++IGNORE_UNRECOGNIZED_ENCODING = "PANTS_IGNORE_UNRECOGNIZED_ENCODING"+RECURSION_LIMIT = "PANTS_RECURSION_LIMIT"+DAEMON_ENTRYPOINT = "PANTS_DAEMON_ENTRYPOINT"+PANTSC_PROFILE = "PANTSC_PROFILE"
Huh, I wonder why it's pantsc and not pantsd.
comment created time in 21 hours
PR opened pantsbuild/pants
Problem
The code implementing pants initialization is somewhat overcomplicated and more verbose than it needs to be.
Solution
This commit:
- Moves the definition of some
PANTS_environment variables relevant to initialization to a common file, to avoid duplicating environment variable string names in different parts of the codebase - Renames
PANTS_ENTRYPOINTtoPANTS_DAEMON_ENTRYPOINT, to make it clearer what role this entrypoint mechanism is playing in pants execution - Folds the fairly-minimal functionality of
pants_exe.pyintopants_loader.pyor integration test files, and removespants_exe.pyentirely
Result
This is a refactoring commit that should have no user-visible change
pr created time in 21 hours
PR opened pantsbuild/pants
WIP; based on https://github.com/pantsbuild/pants/pull/11462
pr created time in a day
pull request commentpantsbuild/pants
Bump Zinc version to support Scala 2.13
@stuhood I've bumped Zinc to the latest compatible version 1.3.5 and registered Scala 2.13 in scala_platform.py. What's the easiest way to publish my branch locally and use it in our project for testing?
comment created time in a day
pull request commentpantsbuild/pants
Allow concurrent runs of pants on the same instance of pantsd
This might be related to https://github.com/pantsbuild/pants/issues/11626, but, I ran ./pants test src/python/pants/base: at the same time in two different tabs. Both correctly rendered the console test summary, but only one showed the std{out,err} of the failed exception_sink_integration_test.py. What would you expect? The EnrichedTestResult rule is uncacheable, but I'm not sure how that plays out with concurrent runs.
To reproduce, probably useful to do a slower test (or add time.sleep()) so it doesn't complete before starting the other run.
comment created time in a day
Pull request review commentpantsbuild/pants
Allow concurrent runs of pants on the same instance of pantsd
function run_pex() { function run_packages_script() { ( cd "${ROOT}"- ./pants --concurrent run "${ROOT}/build-support/bin/packages.py" -- "$@"+ ./pants --no-pantsd run "${ROOT}/build-support/bin/packages.py" -- "$@"
Why this?
comment created time in a day
Pull request review commentpantsbuild/pex
Fix Requires metadata handling.
def test_requires_dists_none(pygoogleearth_zip_sdist): with example_distribution("MarkupSafe-1.0-cp27-cp27mu-linux_x86_64.whl") as (wheel_path, dist): assert [] == list(requires_dists(wheel_path)) assert [] == list(requires_dists(dist))++ # This tests a strange case detailed here:+ # https://github.com/pantsbuild/pex/issues/1201#issuecomment-791715585+ with downloaded_sdist("et-xmlfile==1.0.1") as sdist, warnings.catch_warnings(+ record=True+ ) as events:+ assert [] == list(requires_dists(sdist))+ assert len(events) == 1+ warning = events[0]+ assert PEXWarning == warning.category+ assert (+ dedent(+ """\+ Ignoring 1 `Requires` field in {sdist} metadata:+ 1.) Requires: python (>=2.6.0)++ You may have issues using using the 'et_xmlfile' distribution as a result.
You may have issues using the 'et_xmlfile' distribution as a result.
comment created time in a day
Pull request review commentpantsbuild/pex
Fix Requires metadata handling.
def requires_dists( for requires_dist in pkg_info.get_all("Requires-Dist", ()): yield Requirement.parse(requires_dist) - if include_1_1_requires:- for requires in pkg_info.get_all("Requires", ()):- yield Requirement.parse(requires)+ legacy_requires = pkg_info.get_all("Requires", []) # type: List[str]+ if legacy_requires:+ name_and_version = project_name_and_version(dist)+ project_name = name_and_version.project_name if name_and_version else dist+ pex_warnings.warn(+ dedent(+ """\+ Ignoring {count} `Requires` {field} in {dist} metadata:+ {requires}++ You may have issues using using the '{project_name}' distribution as a result.
You may have issues using the '{project_name}' distribution as a result.
comment created time in a day
Pull request review commentpantsbuild/pants
Require explicit environment usage
def check_and_set(name: str, value: Optional[str]): return FrozenDict(env_var_subset) +@frozen_after_init+@dataclass(unsafe_hash=True)+class EnvironmentRequest:+ """Requests a subset of the variables set in the environment.++ Requesting only the relevant subset of the environment reduces invalidation caused by unrelated+ changes.+ """++ requested: FrozenOrderedSet[str]+ allowed: Optional[FrozenOrderedSet[str]]++ def __init__(self, requested: Sequence[str], allowed: Optional[Sequence[str]] = None):+ self.requested = FrozenOrderedSet(requested)+ self.allowed = None if allowed is None else FrozenOrderedSet(allowed)+++class Environment(FrozenDict[str, str]):+ """A subset of the variables set in the environment."""++ @rule async def complete_environment(session_values: SessionValues) -> CompleteEnvironment: return session_values[CompleteEnvironment] +@rule+async def environment_subset(
Possibly...? I think that a majority of users of Process right now are constructing an environment for the process by hand, and only some cases are pulling it in from the outside world. Most users of InteractiveProcess will use the CompleteEnvironment, but not all, etc.
comment created time in a day
Pull request review commentpantsbuild/pants
Require explicit environment usage
def check_and_set(name: str, value: Optional[str]): return FrozenDict(env_var_subset) +@frozen_after_init+@dataclass(unsafe_hash=True)+class EnvironmentRequest:+ """Requests a subset of the variables set in the environment.++ Requesting only the relevant subset of the environment reduces invalidation caused by unrelated+ changes.+ """++ requested: FrozenOrderedSet[str]+ allowed: Optional[FrozenOrderedSet[str]]++ def __init__(self, requested: Sequence[str], allowed: Optional[Sequence[str]] = None):+ self.requested = FrozenOrderedSet(requested)+ self.allowed = None if allowed is None else FrozenOrderedSet(allowed)+++class Environment(FrozenDict[str, str]):+ """A subset of the variables set in the environment."""++ @rule async def complete_environment(session_values: SessionValues) -> CompleteEnvironment: return session_values[CompleteEnvironment] +@rule+async def environment_subset(
Good point: can drop the async. But the reason to expose this as a @rule is that we want to discourage direct access to the CompleteEnvironment and expose an API that encourages requesting a filtered copy, because it should allow @rules to be cleaned with early-cutoff in more cases. When a @rule requests the entire environment, it will be invalidated for any environment change.
comment created time in 2 days
pull request commentpantsbuild/pants
Allow concurrent runs of pants on the same instance of pantsd
Only the top commit is relevant: the rest is #11641.
comment created time in 2 days
Pull request review commentpantsbuild/pants
Require explicit environment usage
impl Scheduler { command.current_dir(tempdir.path()); } - if hermetic_env {- command.env_clear();- }+ command.env_clear();
Great, that's much better.
comment created time in 2 days
Pull request review commentpantsbuild/pants
Require explicit environment usage
import logging import re-from dataclasses import dataclass-from typing import Dict, Mapping, Optional, Sequence+from typing import Dict, Optional, Sequence from pants.engine.internals.session import SessionValues from pants.engine.rules import collect_rules, rule from pants.util.frozendict import FrozenDict-from pants.util.meta import frozen_after_init logger = logging.getLogger(__name__) name_value_re = re.compile(r"([A-Za-z_]\w*)=(.*)") shorthand_re = re.compile(r"([A-Za-z_]\w*)") -@frozen_after_init-@dataclass(unsafe_hash=True)-class PantsEnvironment:- """PantsEnvironment is a representation of the environment variables the currently-executing- Pants process was invoked with."""+class CompleteEnvironment(FrozenDict):
Good change to subclass.
comment created time in 2 days
Pull request review commentpantsbuild/pants
Allow concurrent runs of pants on the same instance of pantsd
def run_goal_rule( console.flush() return GoalRuleResult(exit_code, stdout.getvalue(), stderr.getvalue()) - def set_options(self, args: Iterable[str], *, env: Mapping[str, str] | None = None) -> None:+ def set_options(+ self, args: Iterable[str], *, env: Mapping[str, str | None] | None = None+ ) -> None: """Update the engine session with new options and/or environment variables. The environment variables will be used to set the `PantsEnvironment`, which is the environment variables captured by the parent Pants process. Some rules use this to be able to read arbitrary env vars. Any options that start with `PANTS_` will also be used to set- options.+ options. Giving an environment variable a value of `None` will cause it to be propagated from+ the test runner's environment (os.environ). This will override any previously configured values. """+ env = {k: (v if v is not None else os.getenv(k, "")) for k, v in env.items()} if env else {}
Extracted to #11641.
comment created time in 2 days
pull request commentpantsbuild/pants
Require explicit environment usage
Commits are useful to review independently.
comment created time in 2 days