profile
viewpoint

alexcrichton/AudioStreamer 71

A streaming audio player class (AudioStreamer) for Mac OS X and iPhone.

alexcrichton/bzip2-rs 33

libbz2 (bzip2 compression) bindings for Rust

alexcrichton/bufstream 29

A buffered I/O stream for Rust

alexcrichton/brotli2-rs 21

Brotli encoders/decoers for Rust

alexcrichton/ars 1

ar in Rust

alexcrichton/atty 1

are you or are you not a tty?

alexcrichton/binaryen 1

Compiler infrastructure and toolchain library for WebAssembly, in C++

alexcrichton/bzip2-ruby 1

Original libbz2 ruby C bindings from Guy Decoux, with some new love

alexcrichton/1password-teams-open-source 0

Get a free 1Password Teams membership for your open source project

push eventalexcrichton/rust-ci-timing-tracker

ci

commit sha be6a325e5d73eac8ba22f8afcd29f6e1a6da0797

init

view details

push time in 3 hours

issue closedbytecodealliance/wat

Remove exception for official test suite for threads

We have an exception right now to skip tests because I was lazy and didn't want to write down all the old names. We should import our own official copy of the test suite or fix this in the crate.

closed time in 9 hours

alexcrichton

delete branch alexcrichton/wat

delete branch : unignore-tests

delete time in 9 hours

push eventbytecodealliance/wat

Alex Crichton

commit sha ca46d208f4a54d9fc39dba522304e2fb0d1cada6

Unignore some passing tests (#58)

view details

push time in 9 hours

PR opened bytecodealliance/wat

Unignore some passing tests
+0 -30

0 comment

1 changed file

pr created time in 9 hours

create barnchalexcrichton/wat

branch : unignore-tests

created branch time in 9 hours

pull request commentbytecodealliance/wat

Implement the wasm annotations proposal

@fitzgen do you mind taking a look at this?

alexcrichton

comment created time in 11 hours

push eventalexcrichton/wat

Alex Crichton

commit sha 91f01d80483df80c0c9d8bc6a911eacd91ec5987

Rustfmt

view details

Alex Crichton

commit sha 3cd94468e3f7f85731039042db62266b42014574

Fix tests and assertions

view details

push time in 11 hours

pull request commentbytecodealliance/wat

Implement the wasm annotations proposal

I'm particularly curious to get some feedback on the implementation here as well. Like parsing comments parsing annotations is really weird because you generally want to alway skip them and ignore them, which makes it very significant and tricky when you want to not skip them. I opted for a pattern such as:

  • When you ask for a "real" token, it skips all comments, whitespace, and annotations.
  • When you ask for an annotation, it skips only whitespace.
  • When you ask for a comment, it skips only whitespace.
  • When you ask a Parser for a specific annotation, it skips all comments, whitespace, and other annotations.

The last point above is intended to be the main usage of annotations in parsers. For example you can see this in the various changes for @name and such here. The latter point is also really weird though because it means that we have to have really wonky code when parsing (module (@name ..)). We want to parse @name but we don't want to skip @custom. I couldn't really think of a general way to express this so I just went ahead and added a bunch of nasty code, hoping this won't come up again.

Another sort of weird point is that parsing an annotation requires parsing the outer parenthesis. All of the idioms in this crate are to have parsers not parser the outer parenthesis, so parsing module fields is also pretty weird. You have to parse @custom annotations first, and then you can go in and parse the module field inside of parenthesis.

There's definitely drawbacks to this approach and I'm not super happy with this. That being said I think it's at least workable and while not as composable and nice we can always see how it works out in practice. One thing I'd like to do is to integrate this into witx and wasm-interface-types first before landing to prove out this design and make sure it's not truly awful to use.

alexcrichton

comment created time in 11 hours

PR opened bytecodealliance/wat

Implement the wasm annotations proposal

This commit implements the wasm annotations proposal in this crate, adding support for a number of features and items:

  • All annotation syntax, e.g. (@foo ...) is now skipped automatically. Parsers will have to explicitly opt-in to parsing annotation syntax, if supported.

  • Support for the @name annotation has been added. This annotation can annotate modules, functions, locals, and params. The name here takes precedent when creating the debug name section, and can support names other than wat identifiers.

  • Support for the @custom annotation has been added as well. This allows definining arbitrary custom sections in the text format. So long as your custom section doesn't refer to binary offsets and such you should be able to eventually round-trip from binary to text and back to binary!

Parsers like witx and wasm-interface-types will need to change their behavior after this update. The previous method of "parsing annotations" will no longer work since annotations are skipped by default.

Closes #14

+1078 -188

0 comment

25 changed files

pr created time in 11 hours

create barnchalexcrichton/wat

branch : annotations

created branch time in 11 hours

pull request commentbytecodealliance/wasmtime

Fix compilation for armv7

An alternative would be to use platform-specific dependencies in consumers, but it's generally not a great experience to do that. If we want to take as a feature that we at least compile for other platforms although return errors because things don't work, I'm also ok with that. (it's just a slight lean after all!)

If we do land this I would prefer to fail "loudly" if we can if this code is attempted to be used though. For example instead of unreachable_unchecked could we panic when we try to get the address of __probestack in the first place? (e.g. during application of relocations)

pepyakin

comment created time in 13 hours

issue openedWebAssembly/annotations

Test assert grammar works, but grammar doesn't fit into current specification

Currently there's a test in this repository that the following parses:

(@a Heiße Würstchen   

)

(pulled from here)

but according to the current proposed text:

annot ::= "(@"idchar+  annotelem* ")"
annotelem ::= keyword | reserved | uN | sN | fN | string | id | "(" annotelem* ")" | "(@"idchar+ annotelem* ")"

I don't think any of these can afford for non-ascii characters such as Heiße. Is it intended that the spec should be updated or should the test case here be dropped?

created time in 14 hours

pull request commentrust-lang/cargo

Add -Zfeatures tracking issues.

@bors: r+

ehuss

comment created time in 15 hours

pull request commentrust-lang/cargo

Use rust-lang/rust linkchecker on CI.

@bors: r+

ehuss

comment created time in 15 hours

delete branch alexcrichton/wat

delete branch : table-init

delete time in 15 hours

PR opened bytecodealliance/wat

Support parsing multi-table `table.init` instruction

With multiple tables you need to be able to specify which table is being initialized! This supports the now-legacy "only specify the element segment" index by assuming table 0 if not specified.

+17 -5

0 comment

4 changed files

pr created time in 18 hours

create barnchalexcrichton/wat

branch : table-init

created branch time in 18 hours

issue openedWebAssembly/reference-types

Binary encoding for `table.init` spec doesn't match implementations

The online specification indicates that the table segment is encoded first and the element segment is encoded second, but this doesn't agree with other implementations, for example:

  • Gecko reads the segment index first which I think also means that the encoding of memory.init doesn't match the specification as well (?)
  • wabt reads the element segment first, then the table index
  • v8 I think also reads the element segment before the table index.

Is this a bug in the specification? Or do engines need to update which one they're reading first?

created time in 18 hours

pull request commentbytecodealliance/wat

doc(ast) Fix an intra link near `Module::resolve`

Thanks!

Hywan

comment created time in 19 hours

push eventbytecodealliance/wat

Ivan Enderlin

commit sha 64b5091a27a4be17883f44437f06f621c80367ab

doc(ast) Fix an intra link near `Module::resolve` (#55) * doc(ast) Fix an intra link near `Module::resolve`. * doc(lib) Deny `intra_doc_link_resolution_failure`. * chore(ast) rustfmt

view details

push time in 19 hours

PR merged bytecodealliance/wat

doc(ast) Fix an intra link near `Module::resolve`

This patch does 2 things:

  1. Fix an intra link in the doc',
  2. Deny intra link failure.
+3 -3

0 comment

2 changed files

Hywan

pr closed time in 19 hours

issue commentbytecodealliance/cranelift

Enable Rust/C functions to "hook up" to multi-value functions

@bnjbvr oh right sorry I realize now how I was saying that things should outright change, but what I meant and a much better way to word it is how you put it, adding support for a different ABI. That'd definitely work well for wasmtime I think!

alexcrichton

comment created time in 19 hours

push eventbytecodealliance/wasmtime

Deploy from CI

commit sha 2dee1f401172cfa31658f5c1403dee55d66ed5f7

Deploy 9c6150b10312e2c4aec27b5885ceba8fb2fc6ce1 to gh-pages

view details

push time in 20 hours

push eventalexcrichton/rust-ci-timing-tracker

ci

commit sha 47d8e7d5b409e67eb43a89baa6628d9b44cf9ed9

init

view details

push time in a day

push eventbytecodealliance/wasmtime

Deploy from CI

commit sha 26d657ab6a8dfad288f8a4c20f1d64617c85e5c9

Deploy 0a020918b5335b17b579c33650023fb53303b376 to gh-pages

view details

push time in a day

PR merged bytecodealliance/wasmtime

Don't let the API fuzz generator run wild

We've got some OOM fuzz test cases getting reported, but these aren't very interesting. The OOMs, after some investigation, are confirmed to be happening because the test is simply allocating thousands of instances with massive tables, quickly exceeding the 2GB memory threshold for fuzzing. This isn't really interesting because this is expected behavior if you instantiate these sorts of modules.

This commit updates the fuzz test case generator to have a "prediction" for each module how much memory it will take to instantiate it. This prediction is then used to avoid instantiating new modules if we predict that it will exceed our memory limit. The limits here are intentionally very squishy and imprecise. The goal here is to still generate lots of interesting test cases, but not ones that simply exhaust memory trivially.

+69 -14

0 comment

1 changed file

alexcrichton

pr closed time in 2 days

delete branch alexcrichton/wasmtime

delete branch : fuzz-limits

delete time in 2 days

push eventbytecodealliance/wasmtime

Alex Crichton

commit sha 0a020918b5335b17b579c33650023fb53303b376

Don't let the API fuzz generator run wild (#959) We've got some OOM fuzz test cases getting reported, but these aren't very interesting. The OOMs, after some investigation, are confirmed to be happening because the test is simply allocating thousands of instances with massive tables, quickly exceeding the 2GB memory threshold for fuzzing. This isn't really interesting because this is expected behavior if you instantiate these sorts of modules. This commit updates the fuzz test case generator to have a "prediction" for each module how much memory it will take to instantiate it. This prediction is then used to avoid instantiating new modules if we predict that it will exceed our memory limit. The limits here are intentionally very squishy and imprecise. The goal here is to still generate lots of interesting test cases, but not ones that simply exhaust memory trivially.

view details

push time in 2 days

issue commentrust-lang/rust

Visibility tokens on types in `extern { ... }` blocks are no longer given to procedural macros

(updated the issue title to reflect what appears to be the actual bug)

alexcrichton

comment created time in 2 days

issue commentrust-lang/rust

Macro-expanded public struct no longer considered public

Ok, wanted to get a gut-check to see if it was known what was happening. Did some reduction and it looks like the macro no longer gets the pub token for the input type.

Given this input:

pub mod wasm {
    #[the_macro::the_macro]
    extern "C" {
        pub type Memory;
    }
}

fn main() {
    let _: wasm::Memory;
}

then with this macro:

extern crate proc_macro;

use proc_macro::*;

#[proc_macro_attribute]
pub fn the_macro(_attr: TokenStream, body: TokenStream) -> TokenStream {
    let g = match body.clone().into_iter().nth(2) {
        Some(TokenTree::Group(g)) => g,
        _ => panic!("unexpected input"),
    };
    let tokens = g.stream().into_iter().collect::<Vec<_>>();
    assert_eq!(tokens[0].to_string(), "pub");
    return body;
}

it fails with:

$ cargo +nightly build
   Compiling wut v0.1.0 (/home/alex/code/wut)
error: custom attribute panicked
 --> src/main.rs:2:5
  |
2 |     #[the_macro::the_macro]
  |     ^^^^^^^^^^^^^^^^^^^^^^^
  |
  = help: message: assertion failed: `(left == right)`
            left: `"type"`,
           right: `"pub"`

error: aborting due to previous error

error: could not compile `wut`.

To learn more, run the command again with --verbose.

On stable it yields the expected error[E0658]: extern types are experimental error which indicates that the pub token was fed to the macro.

If the #[wasm_bindgen] macro doesn't see the pub token it doesn't make the final result public, which is why the privacy error happened in the first place.

alexcrichton

comment created time in 2 days

pull request commentrustwasm/rfcs

Better error handling

Yes wasm-bindgen is already technically memory unsafe due to this issue. It's not really a problem in practice since if you want to catch something you "catch and throw" at the boundary most of the time rather than in the middle of some code. The throw_str and throw_val functions in wasm-bindgen should be (and are not AFAIK) clearly documented that they leak and should be highly discoraged from use.

The general idea as well is that if an exception is thrown and you didn't expect it then you probably won't reenter the wasm, but we also do not place any guards anywhere preventing you from reentering the wasm (but we arguably should).

As far as I know, that's only true if you create things inside of the closure, so try_catch(|| foo(bar)) should be safe, right?

This is both sort of right but also sort of wrong. For example if bar is an owned value then it will leak if foo throws an exception and didn't drop bar first. Furthermore this could easily be accompanied with a lot of stack frames between the try_catch and the actual exit into JS that then caught the exception. I think given how ergonomic try_catch will be it's going to be very easy to get buried deep in a library so that callers don't realize they're leaking destructors on the stack.

I think it's a given the documentation will be clear about this, but I think that I would want to mark it as unsafe in the meantime as well. That will hopefully strongly discourage users from using it deep within libraries, and it would also help get folks reading the documentation to read about the caveats.


As one sort of minor point, we likely don't want this as a blanket try/catch but rather we'll likely want to do some filtering. In discussions at the CG it was pointed out that even the exception handling proposal won't catch wasm traps such as out of bounds reads/writes or stack overflow. We'll want to be sure to not catch these exceptions as well and, if caught, reraise them.

Pauan

comment created time in 2 days

pull request commentbytecodealliance/wasmtime

Pre-generate trampoline functions

Given the recent discussions about memory management and https://github.com/bytecodealliance/wasmtime/issues/960 the only real benefit of this PR is that we pre-generate trampolines all at once which allows us to avoid placing one-per-page. So in that sense this is an optimization which saves memory (but increases compile time), but I'm less certain about the effects of globally caching things and not freeing memory.

With https://github.com/bytecodealliance/wasmtime/issues/960 we honestly might want to cache all trampolines globally in a Store if it's never going to get free'd anyway, that way we'd amortize the cost across instances as well.

alexcrichton

comment created time in 2 days

issue closedbytecodealliance/wasmtime

Dropping a `Module` doesn't free its JIT memory

Running this code for example:

use wasmtime::*;

fn main() {
    let store = Store::default();
    loop {
        drop(Module::new(&store, "(module (func i32.const 0 drop))").unwrap());
    }
}

will cause the program to infinitely consume memory.

I believe the issue here is that compiled code is always appended to a new CodeMemory instance inside of the shared data within a Store, and that's never cleaned up.

closed time in 2 days

alexcrichton

issue commentbytecodealliance/wasmtime

Dropping a `Module` doesn't free its JIT memory

I'm going to close this in favor of https://github.com/bytecodealliance/wasmtime/issues/960 since it's highly related to that sort of memory management, and it sort of sounds like this will be a feature of the API rather than a bug.

alexcrichton

comment created time in 2 days

issue closedbytecodealliance/wasmtime

Management of `*mut VMContext` should go through owned handles

I've been reviewing some code recently and I've always had the sneaking suspicion that we're slinging around too many *mut VMContext handles and not managing reference counts and such well enough. One place in particular I think we're not doing well is with table mangement, for example this code exhibits a use-after-free:

use wasmtime::*;

fn main() -> anyhow::Result<()> {
    let store = Store::default();
    let s = "foo".to_string();
    let f = Func::wrap0(&store, move || {
        println!("{:?}", s);
    });
    let module = Module::new(
        &store,
        r#"
            (module
                (type (func))
                (table (export "table") 1 1 funcref)
                (func (export "foo")
                    i32.const 0
                    call_indirect (type 0)))
        "#,
    )?;

    let instance = Instance::new(&module, &[])?;

    let table = instance.get_export("table").unwrap().table().unwrap();
    table.set(0, f.into())?;
    instance
        .get_export("foo")
        .unwrap()
        .func()
        .unwrap()
        .call(&[])?;
    Ok(())
}

That prints "\u{0}\u{0}\u{0}", and the issue that the f.into() actually ends up releasing the internal allocation at the end of this block because the VMCallerCheckedAnyfunc value only holds *mut VMContext, not InstanceHandle, which would hold a strong reference to the data.

I don't know if there are other instances of this throughout the codebase, but I think that we should start moving towards fewer raw pointers and fewer *mut VMContext values being thrown around wherever possible.

closed time in 2 days

alexcrichton

issue commentbytecodealliance/wasmtime

Management of `*mut VMContext` should go through owned handles

Ok given our call today I'm going to close this in favor of https://github.com/bytecodealliance/wasmtime/issues/960 since the solution would basically be "instances are held alive for as long as the store is alive"

alexcrichton

comment created time in 2 days

issue openedbytecodealliance/wasmtime

Stores cannot deallocate memory until all references are gone

From our discussion today on the wasmtime call the topic of GC, table.set, and https://github.com/bytecodealliance/wasmtime/issues/954 all came up. It was concluded that today with the MVP you can actually create a cyclic dependency between modules, for example:

(module $a
  (table (export "table") 2 2 funcref)
  (func
    i32.const 1
    call_indirect)
  (elem (i32.const 0) func 0))

(module $b
  (import "a" "table" (table 2 2 funcref))
  (func
    i32.const 0
    call_indirect)
  (elem (i32.const 1) func 0))

These two modules only use MVP features but they have a shared table which cyclically references the two modules.

Our conclusion from this was that Store is actually a "clique" of instances together, and once you instantiate something you can never deallocate anything until all the instances have gone. Internally this would entail ensuring that all types exported in wasmtime have some sort of handle on their store (basically all of them already do, this isn't hard).

Implementation-wise I don't think this will be hard to do, but this does have some critical implications I'd like to sort out first:

  • We need to acknowledge that an Instance will never be fully deallocated until all objects referencing the Store have gone away. This feels like a pretty bad crutch to lean on (but it's not like we have much choice) and is something we would need to document thoroughly. You would basically never want user code generating instances into a Store because it means you could OOM quickly. Instead your instance-creation patterns must be known statically. (this may eventually be different with the whole linking proposal, but this would be the status quo that we would have to document)

  • If we start thinking of Store as a "clique of instances" then I don't think the API is quite aligned for that today. For example if you compile a Module I wouldn't expect that to be tied to a particular store. A Module should have its own compiled memory cached, but this can be instantiated into any Store object since it's just adding new references to the compiled memory in multiple stores (possibly). Furthermore I don't know how to rationalize the thread-safety here. For example an Instance cannot be sent across threads (although this is up for debate), but you almost for sure want to share a Module across threads. You may even want multiple threads to be part of a single Store, but if a Store has to have a handle to all of its Instance objects then this is sort of a backwards relation because Store needs to be threadsafe, but it can't be threadsafe because Instance can't be threadsafe.

I think a bunch of this may just boil down to "Alex needs further clarification of what's already there", but I don't really see how Store and Engine map to concepts of what you would want today. I understand they came from v8, but I'm not sure why we want them in our API as well and how it maps to thread safety and such. This is all stuff we need to sort out, though, because the current way we treat multiple stores and instances referencing each other is not memory safe and can easily segfault.

created time in 2 days

issue commentbytecodealliance/wasmtime

What to do about cross-`Store` calls?

Ok so from our discussion on the call today, the answer here appears to be "disallow this", so we'll need to look into guarding all call-sites and such to ensure cross-store objects cannot be mixed at runtime.

alexcrichton

comment created time in 2 days

push eventalexcrichton/wasmtime

Alex Crichton

commit sha b6d67d0a4a94b9bf8adac41cd0335a1500973775

Don't let the API fuzz generator run wild We've got some OOM fuzz test cases getting reported, but these aren't very interesting. The OOMs, after some investigation, are confirmed to be happening because the test is simply allocating thousands of instances with massive tables, quickly exceeding the 2GB memory threshold for fuzzing. This isn't really interesting because this is expected behavior if you instantiate these sorts of modules. This commit updates the fuzz test case generator to have a "prediction" for each module how much memory it will take to instantiate it. This prediction is then used to avoid instantiating new modules if we predict that it will exceed our memory limit. The limits here are intentionally very squishy and imprecise. The goal here is to still generate lots of interesting test cases, but not ones that simply exhaust memory trivially.

view details

push time in 2 days

PR opened bytecodealliance/wasmtime

Don't let the API fuzz generator run wild

We've got some OOM fuzz test cases getting reported, but these aren't very interesting. The OOMs, after some investigation, are confirmed to be happening because the test is simply allocating thousands of instances with massive tables, quickly exceeding the 2GB memory threshold for fuzzing. This isn't really interesting because this is expected behavior if you instantiate these sorts of modules.

This commit updates the fuzz test case generator to have a "prediction" for each module how much memory it will take to instantiate it. This prediction is then used to avoid instantiating new modules if we predict that it will exceed our memory limit. The limits here are intentionally very squishy and imprecise. The goal here is to still generate lots of interesting test cases, but not ones that simply exhaust memory trivially.

+69 -14

0 comment

1 changed file

pr created time in 2 days

push eventalexcrichton/wasmtime

Alex Crichton

commit sha c31ec6d1c862faa565de9a993b41882391e587b5

Don't let the API fuzz generator run wild We've got some OOM fuzz test cases getting reported, but these aren't very interesting. The OOMs, after some investigation, are confirmed to be happening because the test is simply allocating thousands of instances with massive tables, quickly exceeding the 2GB memory threshold for fuzzing. This isn't really interesting because this is expected behavior if you instantiate these sorts of modules. This commit updates the fuzz test case generator to have a "prediction" for each module how much memory it will take to instantiate it. This prediction is then used to avoid instantiating new modules if we predict that it will exceed our memory limit. The limits here are intentionally very squishy and imprecise. The goal here is to still generate lots of interesting test cases, but not ones that simply exhaust memory trivially.

view details

push time in 2 days

create barnchalexcrichton/wasmtime

branch : fuzz-limits

created branch time in 2 days

pull request commentrust-lang/cargo

Build supports a crate-type flag.

Thanks for the PR here! We discussed this a bit yesterday during the cargo triage meeting, and the conclusion was that we're currently not really in a great place to consider this. This issue has a lot of historical precedent scattered throughout various issues in the issue tracker, so we want to be sure to carefully approach this and not quickly try to fix one case without considering others.

From a technical perspective this PR doesn't include any documentation updates, tests, or detailed rationale for why this feature is being designed the way it is. We tried to infer that the motivation for this PR is related to platform-specific crate types (e.g. Android and cdylibs), but that problem space is even larger and also requires careful consideration.

Our conclusion was that it is likely best to approach this feature in an RFC-style format, one where motivation, prior art, detailed design, etc, are all laid out clearly. That should help in figuring out the best way to design this feature!

dvc94ch

comment created time in 2 days

pull request commentrust-lang/cargo

Add new feature resolver.

Alright I'm definitely happy with everything here, so I'm going to r+

I think it's safe to say this has languished long enough (sorry!) and we can continue to iterate in-tree. Thanks again so much for pushing on this @ehuss!

@bors: r+

ehuss

comment created time in 2 days

Pull request review commentrust-lang/cargo

Add new feature resolver.

 pub struct UnitFor {     /// A target for `build.rs` or any of its dependencies, or a proc-macro or     /// any of its dependencies. This enables `build-override` profiles for     /// these targets.-    build: bool,+    host: bool,+    /// A target for a build dependency (or any of its dependencies). This is+    /// used for computing features of build dependencies independently of+    /// other dependency kinds.+    ///+    /// The subtle difference between this and `host` is that the build script+    /// for a non-host package sets this to `false` because it wants the+    /// features of the non-host package (whereas `host` is true because the+    /// build script is being built for the host). `build_dep` becomes `true`+    /// for build-dependencies, or any of their dependencies.+    build_dep: bool,

Ah ok that makes sense, I know I've "used and abused" that CARGO_FEATURE_* is the "same" as #[cfg] in the past myself, so having a special case for this makes sense.

ehuss

comment created time in 2 days

issue closedbytecodealliance/wasmtime

Unsafe function pointers

During module compilation we're casting memory reference to fat pointers to work around problems with borrow checker (here). It's actually unsafe: when the Compiler struct gets dropped, it causes to drop CodeMemory field in it, which drops Mmap in it, which deallocates the code memory. This results in dangling pointers to the functions.

Some ideas to consider:

  1. Leak the executable memory.
  2. Clone reference with appropriate lifetime using unsafe, so we can cheat on borrow checker this way.
  3. Have a newtype for function references with lifetime shorter than 'static.

CC @sunfishcode

closed time in 2 days

mrowqa

issue commentbytecodealliance/wasmtime

Unsafe function pointers

I believe these issues are handled now with the wasmtime crate itself (it internally holds onto all the various allocations as necessary). The internal crates are all documented as pretty unsafe and require pretty deep knowledge of how they work to use them safely, but that's unlikely going to change too too much in the near future, so I'm going to close this as "basically solved at the wasmtime crate layer"

mrowqa

comment created time in 2 days

issue closedbytecodealliance/wasmtime

Relocation of memory.grow and memory.size

As we discussed with @sunfishcode, after merging https://github.com/CraneStation/wasmtime/pull/194 there's probably no need for {Imported,}Memory32{Grow,Size} in RelocationTarget, or at least the tests pass. However, there are is an idea for having a command line setting which allows user to choose between fast startup using cache and less relocations, or better performance paying extra time for compilation and relocation. The mentioned code hasn't been deleted and it's just a tracking issue.

closed time in 2 days

mrowqa

issue commentbytecodealliance/wasmtime

Relocation of memory.grow and memory.size

Thes relocations ended up getting removed in https://github.com/bytecodealliance/wasmtime/pull/949 and we can always make a new issue if the current relocation strategy is too slow for some use cases!

mrowqa

comment created time in 2 days

push eventbytecodealliance/wasmtime

Deploy from CI

commit sha 4c298077e37ad7455db5c4e740c668f041858468

Deploy b96b53eafb4f68720cc562ecd53cfc3542914b15 to gh-pages

view details

push time in 2 days

issue commentbytecodealliance/wasmtime

Small wasm modules taking excessive amounts of time to compile

Ah yeah FWIW I'm testing these and they're all still taking a long amount of time in the register allocator in cranelift, so this is basically just a collection of cases to "see if the new cranelift register allocator is fast" as opposed to new and novel bugs

alexcrichton

comment created time in 2 days

issue commentbytecodealliance/wasmtime

Small wasm modules taking excessive amounts of time to compile

testcase0.wasm.gz

And one more which takes ~1.2s for me, also found through fuzzing

alexcrichton

comment created time in 2 days

issue commentbytecodealliance/wasmtime

Small wasm modules taking excessive amounts of time to compile

testcase0.wasm.gz

Another test case found from fuzzing, this one takes 1.5s to compile locally

alexcrichton

comment created time in 2 days

push eventrustwasm/wasm-bindgen

Ingvar Stepanyan

commit sha d26068dc6ccfc07d0ab1cd2ead85193ddc3880cd

Propagate missing memory argument (#2011) Fixes #2010.

view details

push time in 2 days

PR merged rustwasm/wasm-bindgen

Reviewers
Propagate missing memory argument

Fixes #2010.

+2 -2

1 comment

1 changed file

RReverser

pr closed time in 2 days

issue closedrustwasm/wasm-bindgen

Regression: can't instantiate wasm-bindgen with instance of WebAssembly.Module

Describe the Bug

On master wasm-bindgen trying to instantiate wasm-bindgen with a custom instance of WebAssembly.Module (e.g. in a Cloudflare Worker) results in an error due to maybe_memory being undefined:

image

If you check the generated code with ESLint, it also warns that variable is not defined anywhere:

image

This appears to be a regression, perhaps caused by a refactor in #1996? (cc @Pauan)

closed time in 2 days

RReverser

pull request commentrustwasm/wasm-bindgen

Propagate missing memory argument

Oops, thanks!

RReverser

comment created time in 2 days

pull request commentrustwasm/wasm-bindgen

[WIP] Add support for unstable WebIDL

Ok you can disregard my previous comment about editing webidl files, this looks great!

A few final asks before merging:

  • Can you be sure to add tests to CI that we build web-sys with this feature flag?
  • Can you configure docs.rs to pass this flag when generating documentation?
  • Can all unstable APIs be documented that they need this --cfg flag to get passed when compiling? This could probably reference the documentation in the next bullet.
  • Additionally, can you add documentation about the feature flag? For example how to pass it, when to pass it, how to use RUSTFLAGS, etc.
grovesNL

comment created time in 2 days

push eventbytecodealliance/wasmtime

Deploy from CI

commit sha 3850afccd4617bf546ecfe42ab5f0dcc4135fb2e

Deploy 4460e569cfd56998e1a026ea9e8cc143ab634be8 to gh-pages

view details

push time in 2 days

pull request commentrustwasm/rfcs

Better error handling

Ah ok @Pauan what you've found aligns with my understanding of how things should work, and I can see how an #[inline(never)] tiny shim would "fix" the issue.

I'm a bit wary to rely on such fragile codegen details like this, but it's also probably not the end of the world. I'd want to probably dig in a bit closer with the intricacies of what's happening here at some point before landing this, but I have a hunch that it'd work well enough today to get the job done.

I do think though that there's one other thing worth bringing up which I've forgotten about. Technically it is memory unsafe to unwind stack frames and not execute destructors on the stack frames. For example scoped threads in Rust are only safe because if the stack is unwound then destructors are run. In general it's acceptable to not run destructors in Rust, but if a value is owned by the stack and the stack is unwound it's not memory safe to skip running those destructors. (that's a case where Rust guarantees that destructors are run today)

This is a pretty uncomfortable question for wasm, and one we haven't mostly just ignored up to now. Our general mantra has been "well just catch JS exceptions entering wasm and propagate the Result". With a function like try_catch, however, that makes the error-handling pattern much more prominent and makes it much easier (I think?) to leak resources in Rust, possibly to the detriment of memory safety.

Even without the memory safety question though this makes is very easy to leak resources in Rust which generally wants to be avoided as well. Unfortunately though there's no real way to square this other than "add a full gc" or "wait for the exceptions proposal". I'm curious on your thoughts on this though? Especially wrt the ease that resources can be leaked now.

Pauan

comment created time in 2 days

pull request commentrustwasm/wasm-bindgen

[WIP] Add support for unstable WebIDL

Looks great to me, thanks!

I think that lots of WebSysUnstableApi annotations are still present though? Could the *.webidl be directly imported now except for perhaps a small edit to include an extra mixin to make sure the global property is available?

grovesNL

comment created time in 2 days

pull request commentbytecodealliance/wasmtime

Fix compilation for armv7

I'm personally ambivalent, but would slightly lean towards a preference of not patching up armv7 support like this just yet. I'm curious if others have different opinions though!

pepyakin

comment created time in 2 days

pull request commentrust-lang/backtrace-rs

Define HAVE_KERN_PROC on FreeBSD to fix rust-lang/rust#54434

Ah ok, then this will need to include a submodule update as well to get merged.

t6

comment created time in 2 days

push eventalexcrichton/wasmtime

Alex Crichton

commit sha 95c625deb61b6e127b76d72b5af69d44845444cb

rustfmt

view details

push time in 2 days

Pull request review commentrust-lang/cargo

Add new feature resolver.

 fn deps_of_roots<'a, 'cfg>(roots: &[Unit<'a>], mut state: &mut State<'a, 'cfg>)             } else if unit.target.is_custom_build() {                 // This normally doesn't happen, except `clean` aggressively                 // generates all units.-                UnitFor::new_build()+                UnitFor::new_build(false)

Ok sounds good to me!

ehuss

comment created time in 2 days

Pull request review commentrust-lang/cargo

Add new feature resolver.

+//! Feature resolver.+//!+//! This is a new feature resolver that runs independently of the main+//! dependency resolver. It is intended to make it easier to experiment with+//! new behaviors. When `-Zfeatures` is not used, it will fall back to using+//! the original `Resolve` feature computation. With `-Zfeatures` enabled,+//! this will walk the dependency graph and compute the features using a+//! different algorithm. One of its key characteristics is that it can avoid+//! unifying features for shared dependencies in some situations.+//!+//! The preferred way to engage this new resolver is via+//! `resolve_ws_with_opts`.+//!+//! There are many assumptions made about the resolver itself. It assumes+//! validation has already been done on the feature maps, and doesn't do any+//! validation itself. It assumes dev-dependencies within a dependency have+//! been removed.++use crate::core::compiler::{CompileKind, RustcTargetData};+use crate::core::dependency::{DepKind, Dependency};+use crate::core::resolver::types::FeaturesSet;+use crate::core::resolver::Resolve;+use crate::core::{FeatureValue, InternedString, PackageId, PackageIdSpec, Workspace};+use crate::util::{CargoResult, Config};+use std::collections::{BTreeSet, HashMap, HashSet};+use std::rc::Rc;++/// Map of activated features.+///+/// The key is `(PackageId, bool)` where the bool is `true` if these+/// are features for a build dependency.+type ActivateMap = HashMap<(PackageId, bool), BTreeSet<InternedString>>;++/// Set of all activated features for all packages in the resolve graph.+pub struct ResolvedFeatures {+    activated_features: ActivateMap,+    /// This is only here for legacy support when `-Zfeatures` is not enabled.+    legacy: Option<HashMap<PackageId, Vec<InternedString>>>,+    opts: FeatureOpts,+}++/// Options for how the feature resolver works.+#[derive(Default)]+struct FeatureOpts {+    /// -Zpackage-features, changes behavior of feature flags in a workspace.+    package_features: bool,+    /// -Zfeatures is enabled, use new resolver.+    new_resolver: bool,+    /// Build deps will not share share features with other dep kinds.+    decouple_build_deps: bool,+    /// Dev dep features will not be activated unless needed.+    decouple_dev_deps: bool,+    /// Targets that are not in use will not activate features.+    ignore_inactive_targets: bool,+    /// If enabled, compare against old resolver (for testing).+    compare: bool,+}++impl FeatureOpts {+    fn new(config: &Config, has_dev_units: bool) -> CargoResult<FeatureOpts> {+        let mut opts = FeatureOpts::default();+        let unstable_flags = config.cli_unstable();+        opts.package_features = unstable_flags.package_features;+        let mut enable = |feat_opts: &Vec<String>| {+            opts.new_resolver = true;+            for opt in feat_opts {+                match opt.as_ref() {+                    "build_dep" => opts.decouple_build_deps = true,+                    "dev_dep" => opts.decouple_dev_deps = true,+                    "itarget" => opts.ignore_inactive_targets = true,+                    "all" => {+                        opts.decouple_build_deps = true;+                        opts.decouple_dev_deps = true;+                        opts.ignore_inactive_targets = true;+                    }+                    "compare" => opts.compare = true,+                    "ws" => unimplemented!(),+                    "host" => unimplemented!(),+                    s => anyhow::bail!("-Zfeatures flag `{}` is not supported", s),+                }+            }+            Ok(())+        };+        if let Some(feat_opts) = unstable_flags.features.as_ref() {+            enable(feat_opts)?;+        }+        // This env var is intended for testing only.+        if let Ok(env_opts) = std::env::var("__CARGO_FORCE_NEW_FEATURES") {+            if env_opts == "1" {+                opts.new_resolver = true;+            } else {+                let env_opts = env_opts.split(',').map(|s| s.to_string()).collect();+                enable(&env_opts)?;+            }+        }+        if has_dev_units {+            // Decoupling of dev deps is not allowed if any test/bench/example+            // is being built. It may be possible to relax this in the future,+            // but it will require significant changes to how unit+            // dependencies are computed, and can result in longer build times+            // with `cargo test` because the lib may need to be built 3 times+            // instead of twice.+            opts.decouple_dev_deps = false;+        }+        Ok(opts)+    }+}++/// Features flags requested for a package.+#[derive(Debug, Clone, Eq, PartialEq, Hash)]+pub struct RequestedFeatures {+    pub features: FeaturesSet,+    pub all_features: bool,+    pub uses_default_features: bool,+}++impl RequestedFeatures {+    /// Creates a new RequestedFeatures from the given command-line flags.+    pub fn from_command_line(+        features: &[String],+        all_features: bool,+        uses_default_features: bool,+    ) -> RequestedFeatures {+        RequestedFeatures {+            features: Rc::new(RequestedFeatures::split_features(features)),+            all_features,+            uses_default_features,+        }+    }++    /// Creates a new RequestedFeatures with the given `all_features` setting.+    pub fn new_all(all_features: bool) -> RequestedFeatures {+        RequestedFeatures {+            features: Rc::new(BTreeSet::new()),+            all_features,+            uses_default_features: true,+        }+    }++    fn split_features(features: &[String]) -> BTreeSet<InternedString> {+        features+            .iter()+            .flat_map(|s| s.split_whitespace())+            .flat_map(|s| s.split(','))+            .filter(|s| !s.is_empty())+            .map(InternedString::new)+            .collect::<BTreeSet<InternedString>>()+    }+}++impl ResolvedFeatures {+    /// Returns the list of features that are enabled for the given package.+    pub fn activated_features(&self, pkg_id: PackageId, is_build: bool) -> Vec<InternedString> {+        self.activated_features_int(pkg_id, is_build, true)+    }++    /// Variant of `activated_features` that returns an empty Vec if this is+    /// not a valid pkg_id/is_build combination. Used by `cargo clean` which+    /// doesn't know the exact set.+    pub fn activated_features_unverified(+        &self,+        pkg_id: PackageId,+        is_build: bool,+    ) -> Vec<InternedString> {+        self.activated_features_int(pkg_id, is_build, false)+    }++    fn activated_features_int(+        &self,+        pkg_id: PackageId,+        is_build: bool,+        verify: bool,+    ) -> Vec<InternedString> {+        if let Some(legacy) = &self.legacy {+            legacy.get(&pkg_id).map_or_else(Vec::new, |v| v.clone())+        } else {+            let is_build = self.opts.decouple_build_deps && is_build;+            if let Some(fs) = self.activated_features.get(&(pkg_id, is_build)) {+                fs.iter().cloned().collect()+            } else if verify {+                panic!("features did not find {:?} {:?}", pkg_id, is_build)+            } else {+                Vec::new()+            }+        }+    }+}++pub struct FeatureResolver<'a, 'cfg> {+    ws: &'a Workspace<'cfg>,+    target_data: &'a RustcTargetData,+    /// The platform to build for, requested by the user.+    requested_target: CompileKind,+    resolve: &'a Resolve,+    /// Options that change how the feature resolver operates.+    opts: FeatureOpts,+    /// Map of features activated for each package.+    activated_features: ActivateMap,+    /// Keeps track of which packages have had its dependencies processed.+    /// Used to avoid cycles, and to speed up processing.+    processed_deps: HashSet<(PackageId, bool)>,+}++impl<'a, 'cfg> FeatureResolver<'a, 'cfg> {+    /// Runs the resolution algorithm and returns a new `ResolvedFeatures`+    /// with the result.+    pub fn resolve(+        ws: &Workspace<'cfg>,+        target_data: &RustcTargetData,+        resolve: &Resolve,+        requested_features: &RequestedFeatures,+        specs: &[PackageIdSpec],+        requested_target: CompileKind,+        has_dev_units: bool,+    ) -> CargoResult<ResolvedFeatures> {+        use crate::util::profile;+        let _p = profile::start("resolve features");++        let opts = FeatureOpts::new(ws.config(), has_dev_units)?;+        if !opts.new_resolver {+            // Legacy mode.+            return Ok(ResolvedFeatures {+                activated_features: HashMap::new(),+                legacy: Some(resolve.features_clone()),+                opts,+            });+        }+        let mut r = FeatureResolver {+            ws,+            target_data,+            requested_target,+            resolve,+            opts,+            activated_features: HashMap::new(),+            processed_deps: HashSet::new(),+        };+        r.do_resolve(specs, requested_features)?;+        log::debug!("features={:#?}", r.activated_features);+        if r.opts.compare {+            r.compare();+        }+        Ok(ResolvedFeatures {+            activated_features: r.activated_features,+            legacy: None,+            opts: r.opts,+        })+    }++    /// Performs the process of resolving all features for the resolve graph.+    fn do_resolve(+        &mut self,+        specs: &[PackageIdSpec],+        requested_features: &RequestedFeatures,+    ) -> CargoResult<()> {+        let member_features = self.ws.members_with_features(specs, requested_features)?;+        for (member, requested_features) in &member_features {+            let fvs = self.fvs_from_requested(member.package_id(), requested_features);+            self.activate_pkg(member.package_id(), &fvs, false)?;+        }+        Ok(())+    }++    fn activate_pkg(+        &mut self,+        pkg_id: PackageId,+        fvs: &[FeatureValue],+        for_build: bool,+    ) -> CargoResult<()> {+        // Add an empty entry to ensure everything is covered. This is intended for+        // finding bugs where the resolver missed something it should have visited.+        // Remove this in the future if `activated_features` uses an empty default.+        self.activated_features+            .entry((pkg_id, for_build))+            .or_insert_with(BTreeSet::new);+        for fv in fvs {+            self.activate_fv(pkg_id, fv, for_build)?;+        }+        if !self.processed_deps.insert((pkg_id, for_build)) {

Ah ok the skipping of optional deps below actually does make this sense yeah, sorry I missed that! It still feels a bit funky to me but because it's doing the skip I'm fine deferring to tests to verify the correctness of this. Mind adding a comment here clarifying what sort of recursion this is guarding against, and how more recursion is actually allowed via feature activation below in a limited way?

ehuss

comment created time in 2 days

Pull request review commentrust-lang/cargo

Add new feature resolver.

 impl UnitFor {         }     } -    /// Returns `true` if this unit is for a custom build script or one of its-    /// dependencies.-    pub fn is_build(self) -> bool {-        self.build+    /// Returns a new copy updating it for a build dependency.+    ///+    /// This is part of the machinery responsible for handling feature+    /// decoupling for build dependencies in the new feature resolver.+    pub fn with_build_dep(mut self, build_dep: bool) -> UnitFor {

Eh it's a minor thing, I wouldn't stress out much over it. I definitely know of no naming convention for this, so it may be good to just highlight it in the documentation and move on. (the current call-sites all "look right" which is what matters)

ehuss

comment created time in 2 days

Pull request review commentrust-lang/cargo

Add new feature resolver.

 pub struct UnitFor {     /// A target for `build.rs` or any of its dependencies, or a proc-macro or     /// any of its dependencies. This enables `build-override` profiles for     /// these targets.-    build: bool,+    host: bool,+    /// A target for a build dependency (or any of its dependencies). This is+    /// used for computing features of build dependencies independently of+    /// other dependency kinds.+    ///+    /// The subtle difference between this and `host` is that the build script+    /// for a non-host package sets this to `false` because it wants the+    /// features of the non-host package (whereas `host` is true because the+    /// build script is being built for the host). `build_dep` becomes `true`+    /// for build-dependencies, or any of their dependencies.+    build_dep: bool,

Ok the example looks good to me, thanks! I'm also fine deferring to your thoughts about booleans!

One final question here, though. Technically we have two sets of fetures to deal with, one when the build script is compiled (the --feature flags) and one when it's executed (CARGO_FEATURE_*). Given that is_custom_build is conflating both executing and compiling the build script, I think we may want to be a little more nuanced/precise? I would expect that the compilation of the build script would use the host: true and build_dep: false set of features, but the execution of the build script would use host: false and build_dep: false as well, right? (ok maybe this doesn't have to do with build_dep?)

Also to clarify, in the example you listed here, shared_dep build.rs is only compiled once regardless of feature settings, right? And then it's executed twice maybe depending on feature settings/targets?

ehuss

comment created time in 2 days

push eventbytecodealliance/wasmtime

Deploy from CI

commit sha 968cffd01700d51342c4af83e7037f73b5489bb9

Deploy 80b095f2e28d2cb1cbd9f7fa3e5e388c8e4ae1bc to gh-pages

view details

push time in 2 days

issue commentrustwasm/wasm-bindgen

struct `Memory` is private

Thanks for the report! I've opened https://github.com/rust-lang/rust/issues/69315 since this appears to be a regression in rust-lang/rust.

AurevoirXavier

comment created time in 2 days

issue openedrust-lang/rust

Macro-expanded public struct no longer considered public

I had https://github.com/rustwasm/wasm-bindgen/issues/2009 opened up on a project recently but this looks to be a regression I think. This regressed between nightly-2020-02-19 and nightly-2020-02-20, and bisecting further by commit reveals https://github.com/rust-lang/rust/pull/69271 as the culprit. I'm not sure which of the PRs in that rollup though are the culprit here.

This can be reproduced by running cargo check from the crates/js-sys folder in the wasm-bindgen repository. Unfortunately this one probably isn't super easy to reproduce, so I'm hoping that someone more knowledgeable can hopefully take a look at this and have a better idea about what's going on.

There's definitely macros going on in #[wasm_bindgen] (the Memory type is synthesized by a macro), which leads me to think that https://github.com/rust-lang/rust/pull/69211 may be the culprit? (cc @petrochenkov)

created time in 2 days

pull request commentrust-lang/backtrace-rs

Define HAVE_KERN_PROC on FreeBSD to fix rust-lang/rust#54434

To make sure I understand, this needs extra commits in libbacktrace right? If so the submodule here will need an update before merging this.

t6

comment created time in 2 days

push eventalexcrichton/wasmtime

Alex Crichton

commit sha 4283fdc862ad2158607afb60703c61ed95846f24

Fix a possible use-after-free with `Global` (#956) * Fix a possible use-after-free with `Global` This commit fixes an issue with the implementation of the `wasmtime::Global` type where if it previously outlived the original `Instance` it came from then you could run into a use-after-free. Now the `Global` type holds onto its underlying `InstanceHandle` to ensure it retains ownership of the underlying backing store of the global's memory. * rustfmt

view details

Alex Crichton

commit sha b6be99c9e15c0d4a70cf1e92f5437c6b2e1d466c

Remove memory-related cases from `RelocationTarget` (#949) This commit shrinks the `RelocationTarget` enumeration to remove intrinsic-related relocations since they are no longer used. Instead these function calls are done indirectly via a table in the `VMContext`. This means that all of this is essentially dead code!

view details

Alex Crichton

commit sha 80b095f2e28d2cb1cbd9f7fa3e5e388c8e4ae1bc

Add API to statically assert signature of a `Func` (#955) * Add API to statically assert signature of a `Func` This commit add a family of APIs to `Func` named `getN` where `N` is the number of arguments. Each function will attempt to statically assert the signature of a `Func` and, if matching, returns a corresponding closure which can be used to invoke the underlying function. The purpose of this commit is to add a highly optimized way to enter a wasm module, performing type checks up front and avoiding all the costs of boxing and unboxing arguments within a `Val`. In general this should be much more optimized than the previous `call` API for entering a wasm module, if the signature is statically known. * rustfmt * Remove stray debugging

view details

Alex Crichton

commit sha 7f929ca13476c4460785b20feca6e7ca319c7391

Refactor wasmtime_runtime::Export Instead of an enumeration with variants that have data fields have an enumeration where each variant has a struct, and each struct has the data fields. This allows us to store the structs in the `wasmtime` API and avoid lots of `panic!` calls and various extraneous matches.

view details

Alex Crichton

commit sha 3213448cddc924821032a68cda5dee6242877712

Pre-generate trampoline functions The `wasmtime` crate supports calling arbitrary function signatures in wasm code, and to do this it generates "trampoline functions" which have a known ABI that then internally convert to a particular signature's ABI and call it. These trampoline functions are currently generated on-the-fly and are cached in the global `Store` structure. This, however, is suboptimal for a few reasons: * Due to how code memory is managed each trampoline resides in its own 64kb allocation of memory. This means if you have N trampolines you're using N * 64kb of memory, which is quite a lot of overhead! * Trampolines are never free'd, even if the referencing module goes away. This is similar to #925. * Trampolines are a source of shared state which prevents `Store` from being easily thread safe. This commit refactors how trampolines are managed inside of the `wasmtime` crate and jit/runtime internals. All trampolines are now allocated in the same pass of `CodeMemory` that the main module is allocated into. A trampoline is generated per-signature in a module as well, instead of per-function. This cache of trampolines is stored directly inside of an `Instance`. Trampolines are stored based on `VMSharedSignatureIndex` so they can be looked up from the internals of the `ExportFunction` value. The `Func` API has been updated with various bits and pieces to ensure the right trampolines are registered in the right places. Overall this should ensure that all trampolines necessary are generated up-front rather than lazily. This allows us to remove the trampoline cache from the `Compiler` type, and move one step closer to making `Compiler` threadsafe for usage across multiple threads. Note that as one small caveat the `Func::wrap*` family of functions don't need to generate a trampoline at runtime, they actually generate the trampoline at compile time which gets passed in. Also in addition to shuffling a lot of code around this fixes one minor bug found in `code_memory.rs`, where `self.position` was loaded before allocation, but the allocation may push a new chunk which would cause `self.position` to be zero instead.

view details

Alex Crichton

commit sha 4efac38231f10e1481dc31102097ad71777bf23b

rustfmt

view details

Alex Crichton

commit sha 11712ce4c5102a129cd6df0e5d010d301ef90d02

Fix some merge conflicts

view details

push time in 2 days

delete branch alexcrichton/wasmtime

delete branch : static-assert-func-signature

delete time in 2 days

push eventbytecodealliance/wasmtime

Alex Crichton

commit sha 80b095f2e28d2cb1cbd9f7fa3e5e388c8e4ae1bc

Add API to statically assert signature of a `Func` (#955) * Add API to statically assert signature of a `Func` This commit add a family of APIs to `Func` named `getN` where `N` is the number of arguments. Each function will attempt to statically assert the signature of a `Func` and, if matching, returns a corresponding closure which can be used to invoke the underlying function. The purpose of this commit is to add a highly optimized way to enter a wasm module, performing type checks up front and avoiding all the costs of boxing and unboxing arguments within a `Val`. In general this should be much more optimized than the previous `call` API for entering a wasm module, if the signature is statically known. * rustfmt * Remove stray debugging

view details

push time in 2 days

PR merged bytecodealliance/wasmtime

Add API to statically assert signature of a `Func`

This commit add a family of APIs to Func named getN where N is the number of arguments. Each function will attempt to statically assert the signature of a Func and, if matching, returns a corresponding closure which can be used to invoke the underlying function.

The purpose of this commit is to add a highly optimized way to enter a wasm module, performing type checks up front and avoiding all the costs of boxing and unboxing arguments within a Val. In general this should be much more optimized than the previous call API for entering a wasm module, if the signature is statically known.

+316 -80

0 comment

8 changed files

alexcrichton

pr closed time in 2 days

push eventalexcrichton/wasmtime

Peter Huene

commit sha efc19b593a34680e68058c2f9eb45968df0f28bc

Implement wasm_func_type. This commit implements the missing `wasm_func_type` C API function.

view details

Peter Huene

commit sha 052ae684c2cbbc9da1d81980cd47c722a153a980

Fix memory leak in wasm_importtype_type. This commit fixes a memory leak in `wasm_importtype_type` which returns a non-owned `wasm_externtype_t`.

view details

Alex Crichton

commit sha a09eed97db71e54fe683499faf19a7499020416b

Update wasmparser to 0.51.2 (#952) Fixes a bug in multi-value loop validation, bytecodealliance/wasmparser#195

view details

Peter Huene

commit sha 2fd3a31c06adedd352eaf6fe9e814eb61e41ef9c

Merge pull request #953 from peterhuene/implement-wasm-func-type Implement wasm_func_type in the C API.

view details

Alex Crichton

commit sha b69a061d23a108e2f25be9a7c86ea164aad4c08f

Add a test that segfault handlers ignore non-wasm segfaults (#941) This is the subject of #940 which while fixed is good to have a regression test for!

view details

Alex Crichton

commit sha 4283fdc862ad2158607afb60703c61ed95846f24

Fix a possible use-after-free with `Global` (#956) * Fix a possible use-after-free with `Global` This commit fixes an issue with the implementation of the `wasmtime::Global` type where if it previously outlived the original `Instance` it came from then you could run into a use-after-free. Now the `Global` type holds onto its underlying `InstanceHandle` to ensure it retains ownership of the underlying backing store of the global's memory. * rustfmt

view details

Alex Crichton

commit sha b6be99c9e15c0d4a70cf1e92f5437c6b2e1d466c

Remove memory-related cases from `RelocationTarget` (#949) This commit shrinks the `RelocationTarget` enumeration to remove intrinsic-related relocations since they are no longer used. Instead these function calls are done indirectly via a table in the `VMContext`. This means that all of this is essentially dead code!

view details

Alex Crichton

commit sha 410d27107270526e44770dde9c1c6784a853c37d

Add API to statically assert signature of a `Func` This commit add a family of APIs to `Func` named `getN` where `N` is the number of arguments. Each function will attempt to statically assert the signature of a `Func` and, if matching, returns a corresponding closure which can be used to invoke the underlying function. The purpose of this commit is to add a highly optimized way to enter a wasm module, performing type checks up front and avoiding all the costs of boxing and unboxing arguments within a `Val`. In general this should be much more optimized than the previous `call` API for entering a wasm module, if the signature is statically known.

view details

Alex Crichton

commit sha 0d51863a652cdb0dab062b90251cd59402541cf8

rustfmt

view details

Alex Crichton

commit sha 4bb4086341475d61b6dd7e41852769b8dd421c91

Remove stray debugging

view details

push time in 2 days

Pull request review commentbytecodealliance/wasmtime

Test basic DWARF generation

+#[path = "debug/dump.rs"]

I think it may be best to rename this file to tests/debug/main.rs, and then I think you can remove the #[path] annotations

yurydelendik

comment created time in 2 days

Pull request review commentbytecodealliance/wasmtime

Test basic DWARF generation

+#!/usr/bin/env node++const stdio = { stdio: 'inherit' };

I think this may be a stray var now?

yurydelendik

comment created time in 2 days

pull request commentbytecodealliance/wat

doc(parser) Fix typos

Thanks!

Hywan

comment created time in 2 days

push eventbytecodealliance/wat

Ivan Enderlin

commit sha d31367323ab097c18e2047f01a99c709731f4934

doc(parser) Fix typos (#54) * doc(parser) Fix typos. * doc(parser) Fix a typo around `parens`. * doc(parser) Fix a typo around `parse`.

view details

push time in 2 days

PR merged bytecodealliance/wat

doc(parser) Fix typos

Thanks for the great work!

+3 -3

0 comment

1 changed file

Hywan

pr closed time in 2 days

push eventalexcrichton/rust-ci-timing-tracker

ci

commit sha 677f835abacbe67e7148efebb7c616855a5e9174

init

view details

push time in 2 days

pull request commentrust-lang/cargo

Switch azure to macOS 10.15.

@bors: r+

Alas, such is life where we accumulate lots of workarounds!

ehuss

comment created time in 2 days

push eventbytecodealliance/wasmtime

Deploy from CI

commit sha af1913a861962da550d9f0beeb7baa632a1c59e7

Deploy b6be99c9e15c0d4a70cf1e92f5437c6b2e1d466c to gh-pages

view details

push time in 2 days

push eventbytecodealliance/wasmtime

Deploy from CI

commit sha ad47fd74944b3a1f1ef2ea38e3078d2525e96fd9

Deploy 4283fdc862ad2158607afb60703c61ed95846f24 to gh-pages

view details

push time in 2 days

delete branch alexcrichton/wasmtime

delete branch : shrink-reloc

delete time in 2 days

push eventbytecodealliance/wasmtime

Alex Crichton

commit sha b6be99c9e15c0d4a70cf1e92f5437c6b2e1d466c

Remove memory-related cases from `RelocationTarget` (#949) This commit shrinks the `RelocationTarget` enumeration to remove intrinsic-related relocations since they are no longer used. Instead these function calls are done indirectly via a table in the `VMContext`. This means that all of this is essentially dead code!

view details

push time in 2 days

PR merged bytecodealliance/wasmtime

Remove memory-related cases from `RelocationTarget`

This commit shrinks the RelocationTarget enumeration to remove intrinsic-related relocations since they are no longer used. Instead these function calls are done indirectly via a table in the VMContext. This means that all of this is essentially dead code!

+2 -77

0 comment

5 changed files

alexcrichton

pr closed time in 2 days

delete branch alexcrichton/wasmtime

delete branch : global-fix

delete time in 2 days

push eventbytecodealliance/wasmtime

Alex Crichton

commit sha 4283fdc862ad2158607afb60703c61ed95846f24

Fix a possible use-after-free with `Global` (#956) * Fix a possible use-after-free with `Global` This commit fixes an issue with the implementation of the `wasmtime::Global` type where if it previously outlived the original `Instance` it came from then you could run into a use-after-free. Now the `Global` type holds onto its underlying `InstanceHandle` to ensure it retains ownership of the underlying backing store of the global's memory. * rustfmt

view details

push time in 2 days

PR merged bytecodealliance/wasmtime

Fix a possible use-after-free with `Global`

This commit fixes an issue with the implementation of the wasmtime::Global type where if it previously outlived the original Instance it came from then you could run into a use-after-free. Now the Global type holds onto its underlying InstanceHandle to ensure it retains ownership of the underlying backing store of the global's memory.

+133 -63

0 comment

4 changed files

alexcrichton

pr closed time in 2 days

issue openedbytecodealliance/wasmtime

What to do about cross-`Store` calls?

In reviewing some code today I started wondering what would happen if you started mixing Store values and instances together. For example you can create two Instance objects in two Store objects, what would happen when they're linked?

The concrete things I know of today we have to worry about are:

  • Mainly the Compiler shared state in a Store. This notably contains two fields:
    • signatures is a Store-local registry of all known wasm signatures, mapping them to an index for signature checks during call_indirect. This is actually memory unsafe today because if you mix two Store objects then two different signatures could get the same shared index, meaning call_indirect could call the wrong thing.
    • trap_registry is also a Store-local registry of information about traps. While I don't think this is related to memory safety it does mean that if you're calling code in one instance but started in another the trap in the second instance won't be resolved correctly and we'll get the wrong trap information out of it.
  • There's also maybe issues with the cache but I suspect not, it's not shared state across instances in a Store that I've looked at too too deeply.

I started implementing a fix where we'd simply reject linking instances together if they come from two different Store values, but this is also a problem with any Val::FuncRef getting stored in a table across instances. Especially with reference types this gets really hairy to guard, so I don't think it'll be easy to simply block access at all entry points.

The only fix I can think of is to have a truly global map for all this, but it feels bad to have a truly global ever-expanding map that's never deallocated from. I think we'll want to figure out a way to remove items from the map at least when a Store is dropped (maybe sooner?). In any case wanted to make sure there was an open issue for this!

created time in 3 days

pull request commentrust-lang/cargo

[WIP] Switch azure to macOS 10.15.

In theory the hardware is the same but I wouldn't put it past there being some differences. Other than that the changes to the test suite look good to me, so r=me when you've got the copy thing done.

ehuss

comment created time in 3 days

pull request commentrust-lang/cargo

Better support for license-file.

@bors: r+

Nice!

ehuss

comment created time in 3 days

push eventalexcrichton/wasmtime

Alex Crichton

commit sha b9bf2fdf67aa399d95c5f38031af5367e6925a75

rustfmt

view details

push time in 3 days

Pull request review commentrust-lang/cargo

Add new feature resolver.

+//! Feature resolver.+//!+//! This is a new feature resolver that runs independently of the main+//! dependency resolver. It is intended to make it easier to experiment with+//! new behaviors. When `-Zfeatures` is not used, it will fall back to using+//! the original `Resolve` feature computation. With `-Zfeatures` enabled,+//! this will walk the dependency graph and compute the features using a+//! different algorithm. One of its key characteristics is that it can avoid+//! unifying features for shared dependencies in some situations.+//!+//! The preferred way to engage this new resolver is via+//! `resolve_ws_with_opts`.+//!+//! There are many assumptions made about the resolver itself. It assumes+//! validation has already been done on the feature maps, and doesn't do any+//! validation itself. It assumes dev-dependencies within a dependency have+//! been removed.++use crate::core::compiler::{CompileKind, RustcTargetData};+use crate::core::dependency::{DepKind, Dependency};+use crate::core::resolver::types::FeaturesSet;+use crate::core::resolver::Resolve;+use crate::core::{FeatureValue, InternedString, PackageId, PackageIdSpec, Workspace};+use crate::util::{CargoResult, Config};+use std::collections::{BTreeSet, HashMap, HashSet};+use std::rc::Rc;++/// Map of activated features.+///+/// The key is `(PackageId, bool)` where the bool is `true` if these+/// are features for a build dependency.+type ActivateMap = HashMap<(PackageId, bool), BTreeSet<InternedString>>;++/// Set of all activated features for all packages in the resolve graph.+pub struct ResolvedFeatures {+    activated_features: ActivateMap,+    /// This is only here for legacy support when `-Zfeatures` is not enabled.+    legacy: Option<HashMap<PackageId, Vec<InternedString>>>,+    opts: FeatureOpts,+}++/// Options for how the feature resolver works.+#[derive(Default)]+struct FeatureOpts {+    /// -Zpackage-features, changes behavior of feature flags in a workspace.+    package_features: bool,+    /// -Zfeatures is enabled, use new resolver.+    new_resolver: bool,+    /// Build deps will not share share features with other dep kinds.+    decouple_build_deps: bool,+    /// Dev dep features will not be activated unless needed.+    decouple_dev_deps: bool,+    /// Targets that are not in use will not activate features.+    ignore_inactive_targets: bool,+    /// If enabled, compare against old resolver (for testing).+    compare: bool,+}++impl FeatureOpts {+    fn new(config: &Config, has_dev_units: bool) -> CargoResult<FeatureOpts> {+        let mut opts = FeatureOpts::default();+        let unstable_flags = config.cli_unstable();+        opts.package_features = unstable_flags.package_features;+        let mut enable = |feat_opts: &Vec<String>| {+            opts.new_resolver = true;+            for opt in feat_opts {+                match opt.as_ref() {+                    "build_dep" => opts.decouple_build_deps = true,+                    "dev_dep" => opts.decouple_dev_deps = true,+                    "itarget" => opts.ignore_inactive_targets = true,+                    "all" => {+                        opts.decouple_build_deps = true;+                        opts.decouple_dev_deps = true;+                        opts.ignore_inactive_targets = true;+                    }+                    "compare" => opts.compare = true,+                    "ws" => unimplemented!(),+                    "host" => unimplemented!(),+                    s => anyhow::bail!("-Zfeatures flag `{}` is not supported", s),+                }+            }+            Ok(())+        };+        if let Some(feat_opts) = unstable_flags.features.as_ref() {+            enable(feat_opts)?;+        }+        // This env var is intended for testing only.+        if let Ok(env_opts) = std::env::var("__CARGO_FORCE_NEW_FEATURES") {+            if env_opts == "1" {+                opts.new_resolver = true;+            } else {+                let env_opts = env_opts.split(',').map(|s| s.to_string()).collect();+                enable(&env_opts)?;+            }+        }+        if has_dev_units {+            // Decoupling of dev deps is not allowed if any test/bench/example+            // is being built. It may be possible to relax this in the future,+            // but it will require significant changes to how unit+            // dependencies are computed, and can result in longer build times+            // with `cargo test` because the lib may need to be built 3 times+            // instead of twice.+            opts.decouple_dev_deps = false;+        }+        Ok(opts)+    }+}++/// Features flags requested for a package.+#[derive(Debug, Clone, Eq, PartialEq, Hash)]+pub struct RequestedFeatures {+    pub features: FeaturesSet,+    pub all_features: bool,+    pub uses_default_features: bool,+}++impl RequestedFeatures {+    /// Creates a new RequestedFeatures from the given command-line flags.+    pub fn from_command_line(+        features: &[String],+        all_features: bool,+        uses_default_features: bool,+    ) -> RequestedFeatures {+        RequestedFeatures {+            features: Rc::new(RequestedFeatures::split_features(features)),+            all_features,+            uses_default_features,+        }+    }++    /// Creates a new RequestedFeatures with the given `all_features` setting.+    pub fn new_all(all_features: bool) -> RequestedFeatures {+        RequestedFeatures {+            features: Rc::new(BTreeSet::new()),+            all_features,+            uses_default_features: true,+        }+    }++    fn split_features(features: &[String]) -> BTreeSet<InternedString> {+        features+            .iter()+            .flat_map(|s| s.split_whitespace())+            .flat_map(|s| s.split(','))+            .filter(|s| !s.is_empty())+            .map(InternedString::new)+            .collect::<BTreeSet<InternedString>>()+    }+}++impl ResolvedFeatures {+    /// Returns the list of features that are enabled for the given package.+    pub fn activated_features(&self, pkg_id: PackageId, is_build: bool) -> Vec<InternedString> {+        self.activated_features_int(pkg_id, is_build, true)+    }++    /// Variant of `activated_features` that returns an empty Vec if this is+    /// not a valid pkg_id/is_build combination. Used by `cargo clean` which+    /// doesn't know the exact set.+    pub fn activated_features_unverified(+        &self,+        pkg_id: PackageId,+        is_build: bool,+    ) -> Vec<InternedString> {+        self.activated_features_int(pkg_id, is_build, false)+    }++    fn activated_features_int(+        &self,+        pkg_id: PackageId,+        is_build: bool,+        verify: bool,+    ) -> Vec<InternedString> {+        if let Some(legacy) = &self.legacy {+            legacy.get(&pkg_id).map_or_else(Vec::new, |v| v.clone())+        } else {+            let is_build = self.opts.decouple_build_deps && is_build;+            if let Some(fs) = self.activated_features.get(&(pkg_id, is_build)) {+                fs.iter().cloned().collect()+            } else if verify {+                panic!("features did not find {:?} {:?}", pkg_id, is_build)+            } else {+                Vec::new()+            }+        }+    }+}++pub struct FeatureResolver<'a, 'cfg> {+    ws: &'a Workspace<'cfg>,+    target_data: &'a RustcTargetData,+    /// The platform to build for, requested by the user.+    requested_target: CompileKind,+    resolve: &'a Resolve,+    /// Options that change how the feature resolver operates.+    opts: FeatureOpts,+    /// Map of features activated for each package.+    activated_features: ActivateMap,+    /// Keeps track of which packages have had its dependencies processed.+    /// Used to avoid cycles, and to speed up processing.+    processed_deps: HashSet<(PackageId, bool)>,+}++impl<'a, 'cfg> FeatureResolver<'a, 'cfg> {+    /// Runs the resolution algorithm and returns a new `ResolvedFeatures`+    /// with the result.+    pub fn resolve(+        ws: &Workspace<'cfg>,+        target_data: &RustcTargetData,+        resolve: &Resolve,+        requested_features: &RequestedFeatures,+        specs: &[PackageIdSpec],+        requested_target: CompileKind,+        has_dev_units: bool,+    ) -> CargoResult<ResolvedFeatures> {+        use crate::util::profile;+        let _p = profile::start("resolve features");++        let opts = FeatureOpts::new(ws.config(), has_dev_units)?;+        if !opts.new_resolver {+            // Legacy mode.+            return Ok(ResolvedFeatures {+                activated_features: HashMap::new(),+                legacy: Some(resolve.features_clone()),+                opts,+            });+        }+        let mut r = FeatureResolver {+            ws,+            target_data,+            requested_target,+            resolve,+            opts,+            activated_features: HashMap::new(),+            processed_deps: HashSet::new(),+        };+        r.do_resolve(specs, requested_features)?;+        log::debug!("features={:#?}", r.activated_features);+        if r.opts.compare {+            r.compare();+        }+        Ok(ResolvedFeatures {+            activated_features: r.activated_features,+            legacy: None,+            opts: r.opts,+        })+    }++    /// Performs the process of resolving all features for the resolve graph.+    fn do_resolve(+        &mut self,+        specs: &[PackageIdSpec],+        requested_features: &RequestedFeatures,+    ) -> CargoResult<()> {+        let member_features = self.ws.members_with_features(specs, requested_features)?;+        for (member, requested_features) in &member_features {+            let fvs = self.fvs_from_requested(member.package_id(), requested_features);+            self.activate_pkg(member.package_id(), &fvs, false)?;+        }+        Ok(())+    }++    fn activate_pkg(+        &mut self,+        pkg_id: PackageId,+        fvs: &[FeatureValue],+        for_build: bool,+    ) -> CargoResult<()> {+        // Add an empty entry to ensure everything is covered. This is intended for+        // finding bugs where the resolver missed something it should have visited.+        // Remove this in the future if `activated_features` uses an empty default.+        self.activated_features+            .entry((pkg_id, for_build))+            .or_insert_with(BTreeSet::new);+        for fv in fvs {+            self.activate_fv(pkg_id, fv, for_build)?;+        }+        if !self.processed_deps.insert((pkg_id, for_build)) {+            // Already processed dependencies.+            return Ok(());+        }+        for (dep_pkg_id, deps) in self.deps(pkg_id, for_build) {+            for dep in deps {+                if dep.is_optional() {+                    continue;+                }+                // Recurse into the dependency.+                let fvs = self.fvs_from_dependency(dep_pkg_id, dep);+                self.activate_pkg(+                    dep_pkg_id,+                    &fvs,+                    for_build || (self.opts.decouple_build_deps && dep.is_build()),+                )?;+            }+        }+        Ok(())+    }++    /// Activate a single FeatureValue for a package.+    fn activate_fv(+        &mut self,+        pkg_id: PackageId,+        fv: &FeatureValue,+        for_build: bool,+    ) -> CargoResult<()> {+        match fv {+            FeatureValue::Feature(f) => {+                self.activate_rec(pkg_id, *f, for_build)?;+            }+            FeatureValue::Crate(dep_name) => {+                // Activate the feature name on self.+                self.activate_rec(pkg_id, *dep_name, for_build)?;+                // Activate the optional dep.+                for (dep_pkg_id, deps) in self.deps(pkg_id, for_build) {+                    for dep in deps {+                        if dep.name_in_toml() == *dep_name {

I'm personally a fan of reducing rightward-drift wherever possible so if you're also a fan feel free to switch this to a continue followed by de-indenting the body

ehuss

comment created time in 3 days

more