profile
viewpoint
Georg Brandl birkenfeld FZ Jülich München

alecthomas/importmagic 97

A Python library for finding unresolved symbols in Python code, and the corresponding imports

birkenfeld/advent18 22

Advent of Code 2018 (Rust)

birkenfeld/advent19 2

Advent of Code 2019

birkenfeld/.emacs.d 1

Emacs configuration

birkenfeld/.dotfiles 0

dotfiles repo

birkenfeld/advent15 0

Advent of Code 2015

birkenfeld/advent16 0

Advent of Code 2016

birkenfeld/advent17 0

Advent of Code 2017

issue commentalgesten/ureq

Timeout is reset on stream when calling `into_reader()`

In fact, it seems that timeout_read also has a bug when a read timeout appears in the header part.

Below is a reproducing example (using version 1.5.1 here).

use std::{net, thread, time, io::prelude::*};

fn main() {
    thread::spawn(server);
    thread::sleep(time::Duration::from_millis(100));

    let x = ureq::get("http://localhost:12345/").timeout_read(100).call();
    println!("Response string: {:?}", x.into_string());
}

const WAIT_HEAD: time::Duration = time::Duration::from_millis(0);
const WAIT_BODY: time::Duration = time::Duration::from_millis(0);

fn server() {
    let listener = net::TcpListener::bind(("localhost", 12345)).unwrap();
    let (mut sock, _) = listener.accept().unwrap();
    let mut buf = [0; 128];
    sock.read(&mut buf).unwrap();

    thread::sleep(WAIT_HEAD);
    sock.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 10\r\n").unwrap();
    sock.write_all(b"Connection: close\r\nContent-Type: text/html\r\n\r\n").unwrap();

    thread::sleep(WAIT_BODY);
    sock.write_all(b"xxxxxxxxxx").unwrap();
}

If this is run as given, with no sleeps, everything looks fine:

Response string: Ok("xxxxxxxxxx")

If run with WAIT_BODY set to 1000 milliseconds:

Response string: Ok("xxxxxxxxxx")

instead of a timeout error.

If run with WAIT_HEAD set to 1000 milliseconds:

Response string: Ok("HTTP/1.1 200 OK\r\nContent-Length: 10\r\nConnection: close\r\nContent-Type: text/html\r\n\r\nxxxxxxxxxx")
birkenfeld

comment created time in 23 minutes

issue openedalgesten/ureq

Timeout is reset on stream when calling `into_reader()`

When calling into_reader() on a request, the timeout set with set_timeout_read (NOT the deadline set_timeout) on the request is reset to no timeout by unit::connect() calling DeadlineStream::reset before the socket is returned, and there is no way to set it again.

This makes set_timeout_read basically unused except for reading the headers.

created time in 6 hours

issue closedpygments/pygments

Parsing input inline , is it possible?

I know that we can print the result to stdOut like this: pygmentize -f html source.ext

Nah, is it possible that we can parse code inline? For instance I have PHP code $str='Hello World; and I want to pygmentizing inline like :

pygmentize -f html -l php "$str='Hello World;"

Is it possible?

EDIT: The real question is : "Will this feature be implemented?" since I think this will be very nice if it be implemented.

closed time in 4 days

blackjyn

issue commentpygments/pygments

Parsing input inline , is it possible?

The default source is standard input, so you can use echo "$str='Hello World;" | pygmentize or pygmentize <<<"$str='Hello World;" (here string syntax).

blackjyn

comment created time in 4 days

push eventpygments/pygments.github.io

Anteru

commit sha 77ac4b9e526165e47cc2b0385f550f2964bd53f3

Automated deployment: Wed Oct 14 05:21:40 UTC 2020 3e1b79c82d2df318f63f24984d875fd2a3400808

view details

push time in 6 days

delete branch birkenfeld/pyo3

delete branch : pyiter

delete time in 7 days

issue commentpygments/pygments

2.7.1: tests/examplefiles is missing from PyPI package, but required by test_lexers_other.py

Hi Sandro, for me it looks like the files are shipped in the source tarball. Are you sure you aren't removing them for Debian (because of unclear licenses, I seem to remember...)?

sandrotosi

comment created time in 11 days

Pull request review commentethercat-rs/ethercat

Add util function to fetch SDOs of a slave

+use crate::{Error, Master, SdoEntryAddr, SdoEntryInfo, SdoIdx, SdoInfo, SdoPos, SlavePos, SubIdx};+use std::collections::HashMap;++type Result<T> = std::result::Result<T, Error>;++pub fn slave_sdos(+    master: &mut Master,+    slave_pos: SlavePos,+) -> Result<HashMap<SdoIdx, SdoEntryInfo>> {+    let slave = master.get_slave_info(slave_pos)?;+    let sdo_positions = (0..slave.sdo_count).into_iter().map(SdoPos::from);+    let mut res = HashMap::new();+    for sdo_pos in sdo_positions {+        let SdoInfo {+            idx, max_sub_idx, ..+        } = master.get_sdo(slave_pos, sdo_pos)?;+        let sdo_entries = (0..=u8::from(max_sub_idx))+            .into_iter()+            .map(SubIdx::from)+            .map(|sub_idx| SdoIdx { idx, sub_idx })+            .map(|sdo_idx| (sdo_idx, SdoEntryAddr::ByIdx(sdo_idx)))+            .map(|(sdo_idx, addr)| {+                master+                    .get_sdo_entry(slave_pos, addr)+                    .map(|entry| (sdo_idx, entry))+            })+            .collect::<Result<Vec<_>>>()?;

I guess there's no need to create a Vec just to merge it into the hashmap immediately after...

flosse

comment created time in 12 days

Pull request review commentethercat-rs/ethercat

Add util function to fetch SDOs of a slave

+use crate::{Error, Master, SdoEntryAddr, SdoEntryInfo, SdoIdx, SdoInfo, SdoPos, SlavePos, SubIdx};+use std::collections::HashMap;++type Result<T> = std::result::Result<T, Error>;++pub fn slave_sdos(+    master: &mut Master,+    slave_pos: SlavePos,+) -> Result<HashMap<SdoIdx, SdoEntryInfo>> {+    let slave = master.get_slave_info(slave_pos)?;+    let sdo_positions = (0..slave.sdo_count).into_iter().map(SdoPos::from);+    let mut res = HashMap::new();+    for sdo_pos in sdo_positions {+        let SdoInfo {+            idx, max_sub_idx, ..+        } = master.get_sdo(slave_pos, sdo_pos)?;+        let sdo_entries = (0..=u8::from(max_sub_idx))

Isn't subindex 0 the max-subindex again?

Also, in my experience not all subindices up to max-subindex need to be valid. Is that handled here?

flosse

comment created time in 12 days

Pull request review commentethercat-rs/ethercat

Add util function to fetch SDOs of a slave

+use crate::{Error, Master, SdoEntryAddr, SdoEntryInfo, SdoIdx, SdoInfo, SdoPos, SlavePos, SubIdx};+use std::collections::HashMap;++type Result<T> = std::result::Result<T, Error>;++pub fn slave_sdos(+    master: &mut Master,+    slave_pos: SlavePos,+) -> Result<HashMap<SdoIdx, SdoEntryInfo>> {+    let slave = master.get_slave_info(slave_pos)?;+    let sdo_positions = (0..slave.sdo_count).into_iter().map(SdoPos::from);+    let mut res = HashMap::new();+    for sdo_pos in sdo_positions {+        let SdoInfo {+            idx, max_sub_idx, ..+        } = master.get_sdo(slave_pos, sdo_pos)?;+        let sdo_entries = (0..=u8::from(max_sub_idx))+            .into_iter()+            .map(SubIdx::from)+            .map(|sub_idx| SdoIdx { idx, sub_idx })+            .map(|sdo_idx| (sdo_idx, SdoEntryAddr::ByIdx(sdo_idx)))

Is that separate map really needed? why not construct SdoEntryAddr in the call for get_sdo_entry?

flosse

comment created time in 12 days

Pull request review commentethercat-rs/ethercat

Add util function to fetch SDOs of a slave

+use crate::{Error, Master, SdoEntryAddr, SdoEntryInfo, SdoIdx, SdoInfo, SdoPos, SlavePos, SubIdx};+use std::collections::HashMap;++type Result<T> = std::result::Result<T, Error>;++pub fn slave_sdos(+    master: &mut Master,+    slave_pos: SlavePos,+) -> Result<HashMap<SdoIdx, SdoEntryInfo>> {+    let slave = master.get_slave_info(slave_pos)?;+    let sdo_positions = (0..slave.sdo_count).into_iter().map(SdoPos::from);

into_iter() should be unneeded since range is already an iterator. (Also below)

flosse

comment created time in 12 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventbirkenfeld/pyo3

Georg Brandl

commit sha 4d008218a0a58897bc5a6504a84b9eb2ffb4b591

Add wrapper for PyErr_CheckSignals() to Python. This is a useful API in long-running Rust code, which lets users cancel evaluation by pressing Ctrl-C, and run any other signal handlers that have been set using the signal module.

view details

push time in 12 days

push eventbirkenfeld/pyo3

Georg Brandl

commit sha 818bd6456f71944c49c02d8d1bce25862ab709e0

Add wrapper for PyErr_CheckSignals() to Python. This is a useful API in long-running Rust code, which lets users cancel evaluation by pressing Ctrl-C, and run any other signal handlers that have been set using the signal module.

view details

push time in 12 days

Pull request review commentPyO3/pyo3

Add wrapper for PyErr_CheckSignals() to Python.

 impl<'p> Python<'p> {     pub fn xdecref<T: IntoPyPointer>(self, ptr: T) {         unsafe { ffi::Py_XDECREF(ptr.into_ptr()) };     }++    /// Lets the Python interpreter check for pending signals and invoke the+    /// corresponding signal handlers. This can run arbitrary Python code.+    ///+    /// If an exception is raised by the signal handler, or the default signal+    /// handler raises an exception (such as `KeyboardInterrupt` for `SIGINT`),+    /// an `Err` is returned.

Sure!

birkenfeld

comment created time in 12 days

PullRequestReviewEvent

pull request commentPyO3/pyo3

Add wrapper for PyErr_CheckSignals() to Python.

Well IMO this only has a marginal connection to error handling, and the C version is named unfortunately. It's rather an interpreter related thing.

In newer Pythons, they added APIs with PySignal_ prefix but this didn't exist back then.

birkenfeld

comment created time in 12 days

PR opened rust-lang/this-week-in-rust

fix typo
+1 -1

0 comment

1 changed file

pr created time in 12 days

push eventbirkenfeld/this-week-in-rust

Georg Brandl

commit sha a8b4137fcd3c9fc464d06df33879324989e6857f

fix typo

view details

push time in 12 days

PR opened PyO3/pyo3

Add wrapper for PyErr_CheckSignals() to Python.

I don't know how to test this...

+15 -0

0 comment

1 changed file

pr created time in 13 days

create barnchbirkenfeld/pyo3

branch : signals

created branch time in 13 days

push eventpygments/pygments.github.io

Anteru

commit sha 9d07e446f6b82dc4e0824e0d3796086c0ded5860

Automated deployment: Tue Oct 6 18:45:11 UTC 2020 5d6b610fc2e318ffcab953095ad5ff7db8681337

view details

push time in 14 days

push eventpygments/pygments.github.io

Anteru

commit sha 2258334f145f1c73010a0cbacc9a77bdd742a66b

Automated deployment: Tue Oct 6 18:43:37 UTC 2020 3f300cf4a3f0bc2504ae007222d94d2e44279f41

view details

push time in 14 days

issue closedpygments/pygments

pygmentize -S does not use the styler

Under Linux mint 20: invoking pygmentize -l python mipy.py works correctly, whereas when I try to specify a style (using a name listed by pygmentize -L) pygmentize -S inkpot -l python mipy.py returns the help page

closed time in 14 days

jfmoulin

issue commentpygments/pygments

pygmentize -S does not use the styler

-S is the option to print general style definitions (e.g. CSS) for a given style.

To use a given style, style is a formatter option, e.g. pygmentize -O style=inkpot mipy.py.

jfmoulin

comment created time in 14 days

push eventbirkenfeld/stm_display_rust

Georg Brandl

commit sha 13cc2fc5d38959a9e88562b4a3af55dc893f0e32

implement new plot command that can take multiple y values per point

view details

Georg Brandl

commit sha abc52c8401cc7af6f45cafb4ccdf1d94f1739ae3

fix debian copyright file

view details

Georg Brandl

commit sha a4d465e3d8f37d67ad2ebd99c6890bac81445b86

run test mode only once per version

view details

push time in 14 days

push eventbirkenfeld/stm_display_rust

Georg Brandl

commit sha 09c80f82d1106b97f661d543eee6e7c52b2928c7

dependency update

view details

Jenkins system

commit sha 1278ae79d16920544b8949ffa2a532ccd6d8664f

[deb] Release v1.13.0

view details

Georg Brandl

commit sha 78e4dc8ca67077a2f9fdf3db16221604e4047153

fix build deps

view details

Georg Brandl

commit sha 175fe9f4d095f28d13f48be1866ce9431e092ff1

shrink bounding box of very large font

view details

Georg Brandl

commit sha 8439f4b25372208b72e0f5cb703c0faa26db4f11

simulator: force redraw every second

view details

Georg Brandl

commit sha 764b35516ab65029b72d382e4bb2f14040c9f84b

bump version

view details

Enrico Faulhaber

commit sha ff8875713eaf2b839dd58431442a0b266d7ab49e

fix Makefiles Change-Id: I313101bb2b918c8b5887e3005d4ed8f2ccebcf45 Reviewed-on: https://forge.frm2.tum.de/review/c/frm2/general/display_firmware/+/23494 Tested-by: Enrico Faulhaber <enrico.faulhaber@frm2.tum.de> Reviewed-by: Enrico Faulhaber <enrico.faulhaber@frm2.tum.de>

view details

Georg Brandl

commit sha 2d8eb29405b58cc2a86d840d4e905ce22e3eb132

add new font size inbetween normal and large

view details

Jenkins system

commit sha 8abe754bef2181d378601dcfa713bd946d256f8d

[deb] Release v1.15.0

view details

Enrico Faulhaber

commit sha 9f3fca3ad6dd03eef40358fcdc024b688ef5d682

add 'x_|' to verylarge font Change-Id: I5eca643697d13e895c2ecfc5f1365694fa4c1dce

view details

Enrico Faulhaber

commit sha 5f4e6cf378483171cd04d7350bef19afab89772b

change size of medium font to 12x24 Change-Id: Iad2b0bccfffa0875b6b2f8d9ffbdf2c6f56b2062

view details

Jenkins system

commit sha 8d34219a5d544934102fc5c67f631569118669e5

[deb] Release v1.16.0

view details

Georg Brandl

commit sha 78e6ccadf94d617a176ba82272f3d285eea1932a

Implement some more console features - ignore more control characters - fix backspace erasing stuff - implement more erase/delete escape sequences

view details

Jenkins system

commit sha 3b4e4b7e1ae41bae6da94161c53a48a549a7aa0a

[deb] Release v1.17.0

view details

Georg Brandl

commit sha 9bf056ea8f0af17db136634e7d346ada9f9ad165

Implement even more escape sequences

view details

Jenkins system

commit sha f4b02773dfc3caa2f5b4e2fa6b145c7a8a5e24b8

[deb] Release v1.18.0

view details

Georg Brandl

commit sha 45227213ce184d94dc315559733f8456e2a66ad5

dependency updates

view details

Georg Brandl

commit sha bc66008abab89ffe317418559aae4ff842bfff3a

fix build failure on ARM

view details

push time in 14 days

push eventpygments/pygments.github.io

Anteru

commit sha f18dd587b46f0623a5e234cbac861dd9d035b079

Automated deployment: Tue Oct 6 10:28:58 UTC 2020 9454d5487fc73a427ca8e64c6a6b6330599a600d

view details

push time in 14 days

PR opened gentoo/pax-utils

Fix typo in module docstring
+1 -1

0 comment

1 changed file

pr created time in 15 days

push eventbirkenfeld/pax-utils

Georg Brandl

commit sha ae06ac7b385ed6bb11b7a86c969bf880f42125ef

Fix typo in module docstring

view details

push time in 15 days

fork birkenfeld/pax-utils

[MIRROR] ELF related utils for ELF 32/64 binaries that can check files for security relevant properties

https://gitweb.gentoo.org/proj/pax-utils.git

fork in 15 days

PR closed birkenfeld/rust-systemd

build(deps): bump libc from 0.2.77 to 0.2.79 dependencies

Bumps libc from 0.2.77 to 0.2.79. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/rust-lang/libc/commit/1818abfb3123fa44c8c3dd071635e85d2170647f"><code>1818abf</code></a> libc 0.2.79</li> <li><a href="https://github.com/rust-lang/libc/commit/b4c81e9871e50c1d35ccc973952b1bb71b10aa9e"><code>b4c81e9</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1912">#1912</a> from pfmooney/illumos-pthread-getattr</li> <li><a href="https://github.com/rust-lang/libc/commit/02fe5de20b9d24db107932985aa3b4dc11fe408c"><code>02fe5de</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1911">#1911</a> from joshtriplett/master</li> <li><a href="https://github.com/rust-lang/libc/commit/c4b0d5efd3a3cb17ae830b83bc2abc7ce83be24d"><code>c4b0d5e</code></a> Fix bootstrap on redox</li> <li><a href="https://github.com/rust-lang/libc/commit/8f57688ddec84c4be5064b74fe2b35b62d03fa64"><code>8f57688</code></a> pthread_getattr_np is not present on illumos</li> <li><a href="https://github.com/rust-lang/libc/commit/458f49c8e653344f7392afe9a15b1512d1d76148"><code>458f49c</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1894">#1894</a> from alistair23/alistair/rv32</li> <li><a href="https://github.com/rust-lang/libc/commit/2ae4135cbd9878403b9ed941f659f3216988b177"><code>2ae4135</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1889">#1889</a> from LinkTed/master</li> <li><a href="https://github.com/rust-lang/libc/commit/bdfd01599e0f1962287a9b54b01d05cede3c304b"><code>bdfd015</code></a> unix: riscv32: Move type defines inside confif</li> <li><a href="https://github.com/rust-lang/libc/commit/e3312f5a5ccb6e37f08a60ccc32a7e1cad02eeac"><code>e3312f5</code></a> libc 0.2.78</li> <li><a href="https://github.com/rust-lang/libc/commit/3e9945eb0455585c7296017af88551d4aa7c5ef4"><code>3e9945e</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rust-lang/libc/issues/1897">#1897</a> from Aaron1011/feature/sys-pidfd-open-clone3</li> <li>Additional commits viewable in <a href="https://github.com/rust-lang/libc/compare/0.2.77...0.2.79">compare view</a></li> </ul> </details> <br />

Dependabot compatibility score

You can trigger a rebase of this PR by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

</details>

+2 -2

0 comment

1 changed file

dependabot[bot]

pr closed time in 15 days

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()+++class WatLexer(RegexLexer):+    name = 'WebAssembly'+    aliases = ['wast', 'wat']+    filenames = ['*.wat', '*.wast']++    tokens = {+        'root': [+            (words(keywords, suffix=r'(?=[^a-z_\.])'), Keyword),+            (words(builtins), Name.Builtin, 'arguments'),+            (r'i32.const', Name.Builtin),+            (words(['i32', 'i64', 'f32', 'f64']), Keyword.Type),+            (r'\$[A-Za-z0-9!#$%&\'*+-./:<=>?@\\^_`|~]+', Name.Variable), # yes, all of the are valid in identifiers+            (r';;.*?$', Comment.Single),+            (r'\(;', Comment.Multiline, 'nesting_comment'),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*(.([\dA-Fa-f](_?[\dA-Fa-f])*)?)?([pP][+-]?[\dA-Fa-f](_?[\dA-Fa-f])*)?', Number.Float),+            (r'[+-]?\d.\d(_?\d)*[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?\d.\d(_?\d)*', Number.Float),+            (r'[+-]?\d.[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?(inf|nan|nan:0x[\dA-Fa-f](_?[\dA-Fa-f])*)', Number.Float),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*', Number.Hex),+            (r'[+-]?\d(_?\d)*', Number.Integer),+            (r'[\(\)]', Punctuation),+            (r'"', String.Double, 'string'),+            (r'\s+', Text),+        ],+        'nesting_comment': [+            (r'\(;', Comment.Multiline, '#push'),+            (r';\)', Comment.Multiline, '#pop'),+            (r'(.|\n)', Comment.Multiline)

Ah right, I missed that comments nest. In that case it's

(r'[^;(]+', Comment.Multiline),
(r'[;(]', Comment.Multiline),
jendrikw

comment created time in 16 days

PullRequestReviewEvent

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()

yes, please, but as a tuple (with (...)). This compiles the whole list into a constant without runtime overhead.

jendrikw

comment created time in 16 days

PullRequestReviewEvent

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()+++class WatLexer(RegexLexer):+    name = 'WebAssembly'+    aliases = ['wast', 'wat']+    filenames = ['*.wat', '*.wast']++    tokens = {+        'root': [+            (words(keywords, suffix=r'(?=[^a-z_\.])'), Keyword),+            (words(builtins), Name.Builtin, 'arguments'),+            (r'i32.const', Name.Builtin),+            (words(['i32', 'i64', 'f32', 'f64']), Keyword.Type),+            (r'\$[A-Za-z0-9!#$%&\'*+-./:<=>?@\\^_`|~]+', Name.Variable), # yes, all of the are valid in identifiers+            (r';;.*?$', Comment.Single),+            (r'\(;', Comment.Multiline, 'nesting_comment'),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*(.([\dA-Fa-f](_?[\dA-Fa-f])*)?)?([pP][+-]?[\dA-Fa-f](_?[\dA-Fa-f])*)?', Number.Float),+            (r'[+-]?\d.\d(_?\d)*[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?\d.\d(_?\d)*', Number.Float),+            (r'[+-]?\d.[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?(inf|nan|nan:0x[\dA-Fa-f](_?[\dA-Fa-f])*)', Number.Float),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*', Number.Hex),+            (r'[+-]?\d(_?\d)*', Number.Integer),+            (r'[\(\)]', Punctuation),+            (r'"', String.Double, 'string'),+            (r'\s+', Text),+        ],+        'nesting_comment': [+            (r'\(;', Comment.Multiline, '#push'),+            (r';\)', Comment.Multiline, '#pop'),+            (r'(.|\n)', Comment.Multiline)+        ],+        'string': [+            (r'\\[\dA-Fa-f][\dA-Fa-f]*', String.Escape),+            (r'\t', String.Escape),

these are literal tab, newline and CR. did you mean r'\\t' etc?

jendrikw

comment created time in 16 days

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()+++class WatLexer(RegexLexer):+    name = 'WebAssembly'+    aliases = ['wast', 'wat']+    filenames = ['*.wat', '*.wast']++    tokens = {+        'root': [+            (words(keywords, suffix=r'(?=[^a-z_\.])'), Keyword),+            (words(builtins), Name.Builtin, 'arguments'),+            (r'i32.const', Name.Builtin),+            (words(['i32', 'i64', 'f32', 'f64']), Keyword.Type),+            (r'\$[A-Za-z0-9!#$%&\'*+-./:<=>?@\\^_`|~]+', Name.Variable), # yes, all of the are valid in identifiers+            (r';;.*?$', Comment.Single),+            (r'\(;', Comment.Multiline, 'nesting_comment'),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*(.([\dA-Fa-f](_?[\dA-Fa-f])*)?)?([pP][+-]?[\dA-Fa-f](_?[\dA-Fa-f])*)?', Number.Float),+            (r'[+-]?\d.\d(_?\d)*[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?\d.\d(_?\d)*', Number.Float),+            (r'[+-]?\d.[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?(inf|nan|nan:0x[\dA-Fa-f](_?[\dA-Fa-f])*)', Number.Float),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*', Number.Hex),+            (r'[+-]?\d(_?\d)*', Number.Integer),+            (r'[\(\)]', Punctuation),+            (r'"', String.Double, 'string'),+            (r'\s+', Text),+        ],+        'nesting_comment': [+            (r'\(;', Comment.Multiline, '#push'),+            (r';\)', Comment.Multiline, '#pop'),+            (r'(.|\n)', Comment.Multiline)+        ],+        'string': [+            (r'\\[\dA-Fa-f][\dA-Fa-f]*', String.Escape),+            (r'\t', String.Escape),+            (r'\n', String.Escape),+            (r'\r', String.Escape),+            (r'\\"', String.Escape),+            (r"\\'", String.Escape),+            (r'\\u\{[\dA-Fa-f](_?[\dA-Fa-f])*\}', String.Escape),+            (r'\\\\', String.Escape),+            (chr(92) + chr(92), Error), # backslash (double for regex)+            (r'"', String.Double, '#pop'),+            (r'.', String.Double),+        ],+        'arguments': [+            (r'\s+', Text),+            (r'(offset)(=)(\d(_?\d)*)', bygroups(Keyword, Operator, Number.Integer)),

This will match offset=0x..., so should get moved below the next rule. Similar with align below.

jendrikw

comment created time in 16 days

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()+++class WatLexer(RegexLexer):+    name = 'WebAssembly'+    aliases = ['wast', 'wat']+    filenames = ['*.wat', '*.wast']++    tokens = {+        'root': [+            (words(keywords, suffix=r'(?=[^a-z_\.])'), Keyword),+            (words(builtins), Name.Builtin, 'arguments'),+            (r'i32.const', Name.Builtin),+            (words(['i32', 'i64', 'f32', 'f64']), Keyword.Type),+            (r'\$[A-Za-z0-9!#$%&\'*+-./:<=>?@\\^_`|~]+', Name.Variable), # yes, all of the are valid in identifiers+            (r';;.*?$', Comment.Single),+            (r'\(;', Comment.Multiline, 'nesting_comment'),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*(.([\dA-Fa-f](_?[\dA-Fa-f])*)?)?([pP][+-]?[\dA-Fa-f](_?[\dA-Fa-f])*)?', Number.Float),+            (r'[+-]?\d.\d(_?\d)*[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?\d.\d(_?\d)*', Number.Float),+            (r'[+-]?\d.[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?(inf|nan|nan:0x[\dA-Fa-f](_?[\dA-Fa-f])*)', Number.Float),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*', Number.Hex),+            (r'[+-]?\d(_?\d)*', Number.Integer),+            (r'[\(\)]', Punctuation),+            (r'"', String.Double, 'string'),+            (r'\s+', Text),+        ],+        'nesting_comment': [+            (r'\(;', Comment.Multiline, '#push'),+            (r';\)', Comment.Multiline, '#pop'),+            (r'(.|\n)', Comment.Multiline)+        ],+        'string': [+            (r'\\[\dA-Fa-f][\dA-Fa-f]*', String.Escape),+            (r'\t', String.Escape),+            (r'\n', String.Escape),+            (r'\r', String.Escape),+            (r'\\"', String.Escape),+            (r"\\'", String.Escape),+            (r'\\u\{[\dA-Fa-f](_?[\dA-Fa-f])*\}', String.Escape),+            (r'\\\\', String.Escape),+            (chr(92) + chr(92), Error), # backslash (double for regex)

just r'\\' would be fine, but it's better not to produce Error tokens explicitly. Just replace the final . pattern by:

(r'[^"\\]+', String.Double),

(also speeds up the matching).

jendrikw

comment created time in 16 days

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()+++class WatLexer(RegexLexer):+    name = 'WebAssembly'+    aliases = ['wast', 'wat']+    filenames = ['*.wat', '*.wast']++    tokens = {+        'root': [+            (words(keywords, suffix=r'(?=[^a-z_\.])'), Keyword),+            (words(builtins), Name.Builtin, 'arguments'),+            (r'i32.const', Name.Builtin),

why is this one special?

jendrikw

comment created time in 16 days

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()+++class WatLexer(RegexLexer):+    name = 'WebAssembly'+    aliases = ['wast', 'wat']+    filenames = ['*.wat', '*.wast']++    tokens = {+        'root': [+            (words(keywords, suffix=r'(?=[^a-z_\.])'), Keyword),+            (words(builtins), Name.Builtin, 'arguments'),+            (r'i32.const', Name.Builtin),+            (words(['i32', 'i64', 'f32', 'f64']), Keyword.Type),+            (r'\$[A-Za-z0-9!#$%&\'*+-./:<=>?@\\^_`|~]+', Name.Variable), # yes, all of the are valid in identifiers+            (r';;.*?$', Comment.Single),+            (r'\(;', Comment.Multiline, 'nesting_comment'),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*(.([\dA-Fa-f](_?[\dA-Fa-f])*)?)?([pP][+-]?[\dA-Fa-f](_?[\dA-Fa-f])*)?', Number.Float),+            (r'[+-]?\d.\d(_?\d)*[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?\d.\d(_?\d)*', Number.Float),+            (r'[+-]?\d.[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?(inf|nan|nan:0x[\dA-Fa-f](_?[\dA-Fa-f])*)', Number.Float),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*', Number.Hex),+            (r'[+-]?\d(_?\d)*', Number.Integer),+            (r'[\(\)]', Punctuation),+            (r'"', String.Double, 'string'),+            (r'\s+', Text),+        ],+        'nesting_comment': [+            (r'\(;', Comment.Multiline, '#push'),+            (r';\)', Comment.Multiline, '#pop'),+            (r'(.|\n)', Comment.Multiline)

this can be sped up a lot using:

(r'[^;]+', Comment.Multiline),
(r';', Comment.Multiline),
jendrikw

comment created time in 16 days

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()+++class WatLexer(RegexLexer):

Needs a docstring with version information (see other lexers for examples).

jendrikw

comment created time in 16 days

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()+++class WatLexer(RegexLexer):+    name = 'WebAssembly'+    aliases = ['wast', 'wat']+    filenames = ['*.wat', '*.wast']++    tokens = {+        'root': [+            (words(keywords, suffix=r'(?=[^a-z_\.])'), Keyword),+            (words(builtins), Name.Builtin, 'arguments'),+            (r'i32.const', Name.Builtin),+            (words(['i32', 'i64', 'f32', 'f64']), Keyword.Type),+            (r'\$[A-Za-z0-9!#$%&\'*+-./:<=>?@\\^_`|~]+', Name.Variable), # yes, all of the are valid in identifiers+            (r';;.*?$', Comment.Single),+            (r'\(;', Comment.Multiline, 'nesting_comment'),+            (r'[+-]?0x[\dA-Fa-f](_?[\dA-Fa-f])*(.([\dA-Fa-f](_?[\dA-Fa-f])*)?)?([pP][+-]?[\dA-Fa-f](_?[\dA-Fa-f])*)?', Number.Float),+            (r'[+-]?\d.\d(_?\d)*[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?\d.\d(_?\d)*', Number.Float),+            (r'[+-]?\d.[eE][+-]?\d(_?\d)*', Number.Float),+            (r'[+-]?(inf|nan|nan:0x[\dA-Fa-f](_?[\dA-Fa-f])*)', Number.Float),

The nan:xxx case should come before the nan case, otherwise it will not get matched.

jendrikw

comment created time in 16 days

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()

please change this to a tuple display.

jendrikw

comment created time in 16 days

Pull request review commentpygments/pygments

Fix #1416: add WebAssembly lexer

+# -*- coding: utf-8 -*-+"""+    pygments.lexers.webassembly+    ~~~~~~~~~~~~~~~~~~~++    Lexers for the WebAssembly text format.++    The grammar can be found at https://github.com/WebAssembly/spec/blob/master/interpreter/README.md+    and https://webassembly.github.io/spec/core/text/.+++    :copyright: Copyright 2006-2020 by the Pygments team, see AUTHORS.+    :license: BSD, see LICENSE for details.+"""++from pygments.lexer import RegexLexer, words, bygroups+from pygments.token import Text, Comment, Operator, Keyword, String, Number, Punctuation, Literal, Error, Name++__all__ = ['WatLexer']++keywords = [+    'module', 'import', 'func', 'funcref', 'start', 'param', 'local', 'type',+    'result', 'export', 'memory', 'global', 'mut', 'data', 'table', 'elem',+    'if', 'then', 'else', 'end', 'block', 'loop'+]++builtins = """+unreachable+nop+block+loop+if+else+end+br+br_if+br_table+return+call+call_indirect+drop+select+local.get+local.set+local.tee+global.get+global.set+i32.load+i64.load+f32.load+f64.load+i32.load8_s+i32.load8_u+i32.load16_s+i32.load16_u+i64.load8_s+i64.load8_u+i64.load16_s+i64.load16_u+i64.load32_s+i64.load32_u+i32.store+i64.store+f32.store+f64.store+i32.store8+i32.store16+i64.store8+i64.store16+i64.store32+memory.size+memory.grow+i32.const+i64.const+f32.const+f64.const+i32.eqz+i32.eq+i32.ne+i32.lt_s+i32.lt_u+i32.gt_s+i32.gt_u+i32.le_s+i32.le_u+i32.ge_s+i32.ge_u+i64.eqz+i64.eq+i64.ne+i64.lt_s+i64.lt_u+i64.gt_s+i64.gt_u+i64.le_s+i64.le_u+i64.ge_s+i64.ge_u+f32.eq+f32.ne+f32.lt+f32.gt+f32.le+f32.ge+f64.eq+f64.ne+f64.lt+f64.gt+f64.le+f64.ge+i32.clz+i32.ctz+i32.popcnt+i32.add+i32.sub+i32.mul+i32.div_s+i32.div_u+i32.rem_s+i32.rem_u+i32.and+i32.or+i32.xor+i32.shl+i32.shr_s+i32.shr_u+i32.rotl+i32.rotr+i64.clz+i64.ctz+i64.popcnt+i64.add+i64.sub+i64.mul+i64.div_s+i64.div_u+i64.rem_s+i64.rem_u+i64.and+i64.or+i64.xor+i64.shl+i64.shr_s+i64.shr_u+i64.rotl+i64.rotr+f32.abs+f32.neg+f32.ceil+f32.floor+f32.trunc+f32.nearest+f32.sqrt+f32.add+f32.sub+f32.mul+f32.div+f32.min+f32.max+f32.copysign+f64.abs+f64.neg+f64.ceil+f64.floor+f64.trunc+f64.nearest+f64.sqrt+f64.add+f64.sub+f64.mul+f64.div+f64.min+f64.max+f64.copysign+i32.wrap_i64+i32.trunc_f32_s+i32.trunc_f32_u+i32.trunc_f64_s+i32.trunc_f64_u+i64.extend_i32_s+i64.extend_i32_u+i64.trunc_f32_s+i64.trunc_f32_u+i64.trunc_f64_s+i64.trunc_f64_u+f32.convert_i32_s+f32.convert_i32_u+f32.convert_i64_s+f32.convert_i64_u+f32.demote_f64+f64.convert_i32_s+f64.convert_i32_u+f64.convert_i64_s+f64.convert_i64_u+f64.promote_f32+i32.reinterpret_f32+i64.reinterpret_f64+f32.reinterpret_i32+f64.reinterpret_i64+""".split()+++class WatLexer(RegexLexer):+    name = 'WebAssembly'+    aliases = ['wast', 'wat']+    filenames = ['*.wat', '*.wast']++    tokens = {+        'root': [+            (words(keywords, suffix=r'(?=[^a-z_\.])'), Keyword),+            (words(builtins), Name.Builtin, 'arguments'),+            (r'i32.const', Name.Builtin),+            (words(['i32', 'i64', 'f32', 'f64']), Keyword.Type),+            (r'\$[A-Za-z0-9!#$%&\'*+-./:<=>?@\\^_`|~]+', Name.Variable), # yes, all of the are valid in identifiers

+-. is suspicious, better put the dash at the end of the class.

jendrikw

comment created time in 16 days

PullRequestReviewEvent
PullRequestReviewEvent

pull request commentpygments/pygments

Fix #1416: add WebAssembly lexer

For the empty case, use default('#pop').

jendrikw

comment created time in 16 days

issue commentpygments/pygments

tox and GitHub checks are not aligned

There's no reason for the CI to change anyway.

kurtmckee

comment created time in 16 days

push eventpygments/pygments.github.io

Anteru

commit sha 7ce18093044b9f8c0e842d10aeabbea6b803c2cc

Automated deployment: Sun Oct 4 06:33:23 UTC 2020 91728aedb249ecce18b7077f6e94875b8204a7e5

view details

push time in 16 days

push eventpygments/pygments.github.io

Anteru

commit sha 66b79252bd5c8a25b1bb15d1a9ff09359be16bf4

Automated deployment: Fri Oct 2 16:57:47 UTC 2020 d5f86002609fe1ef04db19aa268fd5b1d95b835d

view details

push time in 18 days

push eventbirkenfeld/.emacs.d

Georg Brandl

commit sha 2ce2c04c7383b105d1bdad76e4650058cebc4fa9

elpa update

view details

Georg Brandl

commit sha 303241e9a7418b8e796c6a9a1a55e1c0066a4001

update rustlib location

view details

push time in 18 days

push eventpygments/pygments.github.io

Anteru

commit sha 2cf5b411b300fac0e31813300b217103098d8c6d

Automated deployment: Wed Sep 30 07:44:59 UTC 2020 a886f74acae50a3016dcb75ba92784e345fd8905

view details

push time in 20 days

issue closedpygments/pygments

tox and GitHub checks are not aligned

After opening #1555 the regex linter warned me about a problem in my patch. This was not caught during extensive testing using tox, and indicates that tox and the GitHub checks are not aligned with each other.

tox must be updated to use the same regex linter that GitHub checks is using so that contributors can catch problems before submitting PR's.

closed time in 21 days

kurtmckee

issue commentpygments/pygments

tox and GitHub checks are not aligned

Got it.

kurtmckee

comment created time in 21 days

IssuesEvent

issue commentpygments/pygments

tox and GitHub checks are not aligned

Uh, that difference should be investigated. It looks like missing parallelism, although user time is significantly higher as well.

kurtmckee

comment created time in 22 days

push eventpygments/pygments.github.io

Anteru

commit sha 62237a8b7367b655fae2b56b9387f00cb2fbbeb1

Automated deployment: Mon Sep 28 18:27:47 UTC 2020 cdf1d488769f57eb926a842d81b976dc9aa86f5b

view details

push time in 22 days

issue commentpygments/pygments

tox and GitHub checks are not aligned

Sure, as long as you can still call the different things individually (i.e. make check, make regexlint etc.)

As for regexlint, I'd prefer to keep using our fork instead of upstream, since we can easily tweak some rules there when needed.

kurtmckee

comment created time in 22 days

issue commentpygments/pygments

tox and GitHub checks are not aligned

There's no need to replace make; even if tox runs the linter (which I approve, of course) the standalone make target is still useful.

kurtmckee

comment created time in 22 days

issue commentpygments/pygments

tox and GitHub checks are not aligned

Regexlint doesn't need make, it's just an external package and you call a script contained there.

I guess you could use regexlint as a library and create a "test" that calls it. It's an abuse of unit tests though, in my opinion.

Or does tox have a facility for integration of other types of checks?

kurtmckee

comment created time in 23 days

issue commentpygments/pygments

MySQL ` (backtick) highlighting

That's the idea, yes.

jord1e

comment created time in a month

issue commentpygments/pygments

MySQL ` (backtick) highlighting

@kurtmckee don't forget that practicality beats purity. The token type names assigned by Pygments don't necessarily have to match the semantic meaning assigned by the language - they're not used by a parser. Usually it's a good idea since similar things will have a similar color for different lexers, but for the use case here it's much easier to use some other token type that already has useful assigned attributes in the various styles.

I don't think it makes sense to introduce new styling definitions in all style classes for a token type that exclusively appears in the MySQL lexer.

jord1e

comment created time in a month

PR opened emberian/this-week-in-rust

Remove the RFC 2582 entry from "New RFCs"

The PR was just a minor fix to the RFC.

+0 -1

0 comment

1 changed file

pr created time in a month

push eventbirkenfeld/this-week-in-rust

Georg Brandl

commit sha 89ce5f1da96afe12c039bceaa656a6c37abdc02f

Remove the RFC 2582 entry from "New RFCs" The PR was just a minor fix to the RFC.

view details

push time in a month

push eventpygments/pygments.github.io

Anteru

commit sha 3179b599e43c3c27b3323d778cef02c0da5d2b98

Automated deployment: Wed Sep 23 16:17:41 UTC 2020 9fca2a106502333db64de61f96c29cd4927c7d84

view details

push time in a month

pull request commentPyO3/pyo3

Update paste requirement from 0.1.6 to 1.0.1

Why didn't the MSRV CI job catch that?

dependabot[bot]

comment created time in a month

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 def get_tokens_unprocessed(self, text):     def analyse_text(text):         """Not much we can go by. (* for comments is our best guess."""         result = 0++        is_pascal_like = 0         if '(*' in text and '*)' in text:-            result += 0.01+            is_pascal_like += 0.5         if ':=' in text:-            result += 0.01+            is_pascal_like += 0.5++        if is_pascal_like == 1:

Looks good, but can you make this a boolean flag (pascal = x in text and y in text and z in text)? The float-adding-and-comparing seems to be unneeded...

Anteru

comment created time in a month

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 def analyse_text(text):         decent amount of Tamil-characters, it's this language. This assumption         is obviously horribly off if someone uses string literals in tamil         in another language."""-        if len(re.findall('[\u0b80-\u0bff]')) > 10:+        if len(re.findall(r'[\u0b80-\u0bff]', text)) > 10:

Oops, I missed that.

Anteru

comment created time in a month

PullRequestReviewEvent
PullRequestReviewEvent

pull request commentjaparic/heapless

defmt impl for Vec

If i understand what defmt is doing correctly, for strings they are compile-time interned and arn't actually sent over-the-wire.

Well that's for format strings and other constant strings. But there are also primitive types "byte slice" and "string slice" which are sent as-is (see https://defmt.ferrous-systems.com/primitives.html). I think you just need to call fmt.str() on the Formatter and that should be all.

theunkn0wn1

comment created time in a month

push eventpygments/pygments.github.io

birkenfeld

commit sha 002ed478d0a056a340be98d1b9836fa9cf28f909

Automated deployment: Wed Sep 23 03:29:28 UTC 2020 f0da3b90e8eda5feb58aac2d5daa9921a88a3502

view details

push time in a month

pull request commentpygments/pygments

Grammar correction 'an generator' -> 'a generator'

thanks!

zkneupper

comment created time in a month

push eventpygments/pygments

zkneupper

commit sha f0da3b90e8eda5feb58aac2d5daa9921a88a3502

Grammar correction 'an generator' -> 'a generator'

view details

push time in a month

PR merged pygments/pygments

Grammar correction 'an generator' -> 'a generator'

Corrected grammar in docstring for get_all_styles() function.

Original: """Return an generator for all styles by name, ...

New: """Return a generator for all styles by name, ..

+1 -1

0 comment

1 changed file

zkneupper

pr closed time in a month

PR opened python/cpython

bpo-41825: restructure docs for the os.wait*() family

Mostly fixes up the specific docs of waitid, so that basic usage and all flags are explained without having to consult the manpage.

While doing that, I noticed a few issues with the other wait* functions. I tried to reorder them and W* constants to be sequential, and clearly mark which flag can be used for which function.

+139 -92

0 comment

2 changed files

pr created time in a month

push eventbirkenfeld/cpython

Georg Brandl

commit sha ad37fbf3d285f35513b9baec3de8a0be44d54361

bpo-41825: restructure docs for the os.wait*() family

view details

push time in a month

push eventbirkenfeld/cpython

Raymond Hettinger

commit sha f01d1be97d740ea0369379ca305646a26694236e

bpo-40286: Put methods in correct sections. Add security notice to use secrets for session tokens. (GH-19870)

view details

Victor Stinner

commit sha 4e01946cafca0cf49f796c3118e0d65237bcad69

bpo-40513: Per-interpreter signals pending (GH-19924) Move signals_pending from _PyRuntime.ceval to PyInterpreterState.ceval.

view details

Victor Stinner

commit sha 0b1e3307e24b0af45787ab6456535b8346e0239a

bpo-40513: Per-interpreter gil_drop_request (GH-19927) Move gil_drop_request member from _PyRuntimeState.ceval to PyInterpreterState.ceval.

view details

Victor Stinner

commit sha c5fa364f4ea836f25dd07cfb328152d40a568371

bpo-40514: Add --with-experimental-isolated-subinterpreters (GH-19926) Add --with-experimental-isolated-subinterpreters build option to configure: better isolate subinterpreters, experimental build mode. When used, force the usage of the libc malloc() memory allocator, since pymalloc relies on the unique global interpreter lock (GIL).

view details

Javier Buzzi

commit sha 627f7012353411590434a7d5777ddcbcc8d97fcd

bpo-32117: Updated Simpsons names in docs (GH-19737) `sally` is not a Simpsons character Automerge-Triggered-By: @gvanrossum

view details

Victor Stinner

commit sha 4e30ed3af06ae655f4cb8aad8cba21f341384250

bpo-40513: Per-interpreter recursion_limit (GH-19929) Move recursion_limit member from _PyRuntimeState.ceval to PyInterpreterState.ceval. * Py_SetRecursionLimit() now only sets _Py_CheckRecursionLimit of ceval.c if the current Python thread is part of the main interpreter. * Inline _Py_MakeEndRecCheck() into _Py_LeaveRecursiveCall(). * Convert _Py_RecursionLimitLowerWaterMark() macro into a static inline function.

view details

Victor Stinner

commit sha b0be6b3b94fbdf31b796adc19dc86a04a52b03e1

bpo-29587: _PyErr_ChainExceptions() checks exception (GH-19902) _PyErr_ChainExceptions() now ensures that the first parameter is an exception type, as done by _PyErr_SetObject(). * The following function now check PyExceptionInstance_Check() in an assertion using a new _PyBaseExceptionObject_cast() helper function: * PyException_GetTraceback(), PyException_SetTraceback() * PyException_GetCause(), PyException_SetCause() * PyException_GetContext(), PyException_SetContext() * PyExceptionClass_Name() now checks PyExceptionClass_Check() with an assertion. * Remove XXX comment and add gi_exc_state variable to _gen_throw(). * Remove comment from test_generators

view details

Hai Shi

commit sha 6351d9e4400a77fe1fcbe4f03e5fb6620cca236d

bpo-40520: Remove redundant comment in pydebug.h (GH-19931) Automerge-Triggered-By: @corona10

view details

Victor Stinner

commit sha 299b8c61e9d1a42b929b8deb1b05067876e191e6

Revert "bpo-40513: Per-interpreter signals pending (GH-19924)" (GH-19932) This reverts commit 4e01946cafca0cf49f796c3118e0d65237bcad69.

view details

Victor Stinner

commit sha 607b1027fec7b4a1602aab7df57795fbcec1c51b

bpo-40521: Disable Unicode caches in isolated subinterpreters (GH-19933) When Python is built in the experimental isolated subinterpreters mode, disable Unicode singletons and Unicode interned strings since they are shared by all interpreters. Temporary workaround until these caches are made per-interpreter.

view details

Steve Dower

commit sha ac4bf424119d1300f57929120968e216a85d3a25

bpo-40458: Increase reserved stack space to prevent overflow crash on Windows (GH-19845)

view details

Victor Stinner

commit sha b4b53868d7d6cd13505321d3802fd00865b25e05

bpo-40521: Disable free lists in subinterpreters (GH-19937) When Python is built with experimental isolated interpreters, disable tuple, dict and free free lists. Temporary workaround until these caches are made per-interpreter. Add frame_alloc() and frame_get_builtins() subfunctions to simplify _PyFrame_New_NoTrack().

view details

Victor Stinner

commit sha e838a9324c1719bb917ca81ede8d766b5cb551f4

bpo-40522: _PyThreadState_Swap() sets autoTSSkey (GH-19939) In the experimental isolated subinterpreters build mode, _PyThreadState_GET() gets the autoTSSkey variable and _PyThreadState_Swap() sets the autoTSSkey variable. * Add _PyThreadState_GetTSS() * _PyRuntimeState_GetThreadState() and _PyThreadState_GET() return _PyThreadState_GetTSS() * PyEval_SaveThread() sets the autoTSSkey variable to current Python thread state rather than NULL. * eval_frame_handle_pending() doesn't check that _PyThreadState_Swap() result is NULL. * _PyThreadState_Swap() gets the current Python thread state with _PyThreadState_GetTSS() rather than _PyRuntimeGILState_GetThreadState(). * PyGILState_Ensure() no longer checks _PyEval_ThreadsInitialized() since it cannot access the current interpreter.

view details

Victor Stinner

commit sha 0dd5e7a718997da2026ed64fe054dc36cae4fee7

bpo-40513: new_interpreter() init GIL earlier (GH-19942) Fix also code to handle init_interp_main() failure.

view details

Victor Stinner

commit sha 7be4e350aadf93c4be5c97b7291d0db2b6bc1dc4

bpo-40513: Per-interpreter GIL (GH-19943) In the experimental isolated subinterpreters build mode, the GIL is now per-interpreter. Move gil from _PyRuntimeState.ceval to PyInterpreterState.ceval. new_interpreter() always get the config from the main interpreter.

view details

Victor Stinner

commit sha fb2c7c4afbab0514352ab0246b0c0cc85d1bba53

bpo-40513: _xxsubinterpreters.run_string() releases the GIL (GH-19944) In the experimental isolated subinterpreters build mode, _xxsubinterpreters.run_string() now releases the GIL.

view details

Curtis Bucher

commit sha c21c51235aa8061da6b0593d6f857f42fd92fd8b

bpo-40355: Improve error messages in ast.literal_eval with malformed Dict nodes (GH-19868) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

view details

Dennis Sweeney

commit sha 1253c3ef70ea5632d32ae19579a14152db0d45c1

bpo-40504: Allow weakrefs to lru_cache objects (GH-19938)

view details

Pablo Galindo

commit sha 96074de573f82fc66a2bd73c36905141a3f1d5c1

bpo-40523: Add pass-throughs for hash() and reversed() to weakref.proxy objects (GH-19946)

view details

Tim Peters

commit sha b9c46a2c2d7fc68457bff641f78932d66f5e5f59

bpo-40480 "fnmatch" exponential execution time (GH-19908) bpo-40480: create different regexps in the presence of multiple `*` patterns to prevent fnmatch() from taking exponential time.

view details

push time in a month

create barnchbirkenfeld/cpython

branch : fix-waitid-docs

created branch time in a month

PullRequestReviewEvent

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 class BrainfuckLexer(RegexLexer):         ]     } +    def analyse_text(text):+        """It's safe to assume that any reasonable brainfuck program will have+        more than two consecutive + signs, which is very uncommon in any+        other programming language."""+        if '+++++' in text or '------' in text:+            return 0.5+        if '+++' in text:

While this is probably a good test, I'm worrying about the performance implications of looping over the string on the Python level... the other tests at least use the C-based regex/stringlib routines.

Anteru

comment created time in a month

pull request commentstm32-rs/stm32f4xx-hal

add USART example

Why not show receiving data as well? That's usually the more tricky part...

nebelgrau77

comment created time in a month

push eventpygments/pygments.github.io

Anteru

commit sha bc44148cfa0a52ff93b71f9afc716d7e96af6e0a

Automated deployment: Sun Sep 20 08:27:17 UTC 2020 8aa2bc3d6c1bc65fbc6da9af8a588fcea3552e98

view details

push time in a month

push eventpygments/pygments.github.io

birkenfeld

commit sha 21ecc2fab637ec65ff6031f1b24445073a6dc102

Automated deployment: Sun Sep 20 06:31:16 UTC 2020 45b3dc6891c6ece14e0ea5e017624fef36c6beea

view details

push time in a month

push eventpygments/pygments

Georg Brandl

commit sha 45b3dc6891c6ece14e0ea5e017624fef36c6beea

image formatter: find ttc fonts on Mac fixes #1223

view details

push time in a month

issue closedpygments/pygments

allow to load .ttc fonts

(Original issue 1519 created by vanyakosmos on 2019-06-01T18:04:00.912951+00:00)

formatter.img:123

instead of loading just ttf:

def _create_mac(self):
    font_map = {}
    for font_dir in (os.path.join(os.getenv('HOME'), 'Library/Fonts/'),
                      '/Library/Fonts/', '/System/Library/Fonts/'):
        font_map.update(
            ((os.path.splitext(f)[0].lower(), os.path.join(font_dir, f))
                for f in os.listdir(font_dir) if f.lower().endswith('ttf')))

check multiple extensions:

def _create_mac(self):
    font_map = {}
    for font_dir in (
        os.path.join(os.getenv('HOME'), 'Library/Fonts/'),
        '/Library/Fonts/', 
        '/System/Library/Fonts/',
    ):
        font_map.update(
            (os.path.splitext(f)[0].lower(), os.path.join(font_dir, f))
            for f in os.listdir(font_dir)
            if f.lower().endswith(('ttf', 'ttc'))
        )

closed time in a month

Anteru

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 class HybrisLexer(RegexLexer):         ],     } +    def analyse_text(text):+        """public method and private method don't seem to be quite common+        elsewhere."""+        result = 0+        if re.match(r'(?:public|private)\s+method', text):

match

Anteru

comment created time in a month

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 def get_tokens_unprocessed(self, text):              # return result             yield index, token, value++    def analyse_text(text):+        """Not much we can go by. (* for comments is our best guess."""

that's the same for all pascal-like languages, unfortunately.

Anteru

comment created time in a month

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 class SingularityLexer(RegexLexer):             (r'(.+?(?=^\s*%))|(.*)', using(BashLexer), '#pop'),         ],     }++    def analyse_text(text):+        """This is a quite simple script file, but there are a few keywords+        which seem unique to this language."""+        if re.match('osversion|includecmd', text, re.IGNORECASE):

match is anchored at the beginning of the text, is that intended?

Anteru

comment created time in a month

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 class OctaveLexer(RegexLexer):         ],     } +    def analyse_text(text):+        """Octave is quite hard to spot from a casual glance, but we'll assume+        that most examples will have an integer vector or matrix somewhere."""+        # A \ B is actually quite uncommon outside of Matlab/Octave+        if re.match(r'\w+\s*\\\s*\w+',text):+            return 0.05+        if re.match(r'\[\s*(?:(?:\d+\s*)+;\s*)+\s*\]', text):

This is a quite complicated regex that might backtrack (not catastrophically, but still a bit much to do for analyse_text...)

Anteru

comment created time in a month

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 def get_tokens_unprocessed(self, text):         while objectloop_queue:             yield objectloop_queue.pop(0) +    def analyse_text(text):+        """We try to find a keyword which seem relatively common, unfortunately+        there is a decent overlap with Smalltalk keywords otherwise here.."""+        result = 0+        if re.match('origsource', text, re.IGNORECASE):

match vs search again?

Anteru

comment created time in a month

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 class BrainfuckLexer(RegexLexer):         ]     } +    def analyse_text(text):+        """It's safe to assume that any reasonable brainfuck program will have+        more than two consecutive + signs, which is very uncommon in any+        other programming language."""+        if '+++++' in text or '------' in text:+            return 0.5+        if '+++' in text:

+++ and --- are common in diff/patch files, which are probably more common than brainfuck :)

The longer +++++++ and ------ sequences are typical for underlines in readable markup like rst/markdown.

Another signature of brainfuck which has less conflicts is probably [-], i.e. "clear this cell".

Anteru

comment created time in a month

Pull request review commentpygments/pygments

Add analyze_text to make make check happy.

 class FSharpLexer(RegexLexer):             (r'"', String),         ],     }++    def analyse_text(text):+        """F# doesn't have that many unique features -- |> for matching+        seems quite common though, in addition to let/match."""

let/match/-> are quite common in functional and derived languages, e.g. Haskell or Rust. I wouldn't use them.

Anteru

comment created time in a month

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentpygments/pygments

Add a check for CR/LF in files.

 export PYTHONPATH = $(shell echo "$$PYTHONPATH"):$(shell python -c 'import os; p all: clean-pyc check test  check:+	@$(PYTHON) scripts/check_crlf.py pygments build external || true

It's shell syntax - || true will ignore the exit status of the script and exit with success. It's used for the other invocations because they are (at the moment) producing errors.

Once you fix all the warnings you can remove this for all the other lines so that they actually do complain if something is wrong :)

Anteru

comment created time in a month

PullRequestReviewEvent
more