profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/fwip/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

Altius/hotspot2 12

Implementation of hotspot2 by Eric Rynes

fwip/autoswarm 11

if you glued together automerge and hyperswarm - you get an autoswarm

fwip/dat-fuse 7

A proof-of-concept FUSE filesystem for datproject.org

Altius/Index 4

Methods and code for deriving an Index of DNaseI hypersensitive sites

fwip/earthstar-fuse 2

proof-of-concept implementation of FUSE for Earthstar

fwip/bumblebee 1

a lil' p2p VR experiment

fwip/earthstar-go 1

work in progress implementation of earthstar in go

fwip/2cansavior 0

Savior posts on twocansandstring

fwip/adapter_trimmer 0

WIP tool for trimming sequencing adapters

create barnchStamLab/stampipes

branch : feat/workdir_cleanup_and_locking

created branch time in 2 days

startedstereobooster/awesome-vscode-dev-containers

started time in 4 days

issue commentStamLab/stampipes

docs: bamcounts.py header comment is wrong

Actually most of the terminology is wrong

fwip

comment created time in 6 days

issue openedStamLab/stampipes

docs: bamcounts.py header comment is wrong

This script does not filter, it counts.

created time in 6 days

startedzpaq/zpaq

started time in 7 days

pull request commentreturntocorp/semgrep

RFC: Add new default metrics collection when pulling from server

Speaking as a casual semgrep user:

Sounds good to me. You'll almost always get pushback when moving from opt-in to opt-out, so I appreciate that you've added a "first-run" notice. You might consider linking to the metrics documentation in that message as well, just so that users feel like they have enough information to trust you.

nbrahms

comment created time in 13 days

startedpandaman64/effective-rust

started time in 14 days

issue openedreturntocorp/semgrep

Autofix: Issue with replacing python tuples?

Slack discussion: https://r2c-community.slack.com/archives/C018NJRRCJ0/p1631066079031800

Describe the bug Difficulty matching tuples/parenthesized expressions (?) when writing autofix rules for python. To my eyes, it appears that the trailing ) or ,)) written in the pattern are not matched/replaced when using the autofix feature. I would like to figure out how to either capture these in the metavariables or match&discard them.

To Reproduce https://semgrep.dev/s/fwip:wip-permalink-removal

Expected behavior The output should have the specified number of parentheses and remain syntactically-valid python code.

Screenshots The green arrows in this screenshot point to the part of the pattern that is not getting captured/replaced. Screen Shot 2021-09-07 at 7 43 05 PM

What is the priority of the bug to you?

  • [ ] P0: blocking your adoption of Semgrep or workflow
  • [ ] P1: important to fix or quite annoying
  • [X] P2: regular bug that should get fixed

Environment If not using semgrep.dev: are you running off docker, an official binary, a local build?

semgrep.dev and local on mac via brew (version 0.64.0).

created time in 21 days

startedinformationsea/autocompress-rs

started time in 23 days

push eventStamLab/stampipes

Jemma Nelson

commit sha cef178d3cb670675240acc63f5e1fa2dd5d272f8

bamcounts.py - use nuclear_chroms file in workflow

view details

push time in a month

create barnchStamLab/stampipes

branch : feat/accurate-nuclear-counts

created branch time in a month

create barnchStamLab/stampipes

branch : feat/dnase-dsl2

created branch time in a month

push eventStamLab/stampipes

solexa

commit sha 9a39250e569b634d3ee6d07e311dde20847f1af3

Load openssl-dev module for python upload

view details

push time in a month

push eventStamLab/stampipes

solexa

commit sha aa53e410a0a7c237d5e8ad8aa2a62e99d6ce6041

Fixes for featurea counts & anaquin stats reporting

view details

solexa

commit sha 45e0369edee3130a1920f9bf6e58e13815c5f8ae

Add missing 'module purge' directive to RNA

view details

solexa

commit sha e65d0d7ecad72ae97cf7fc7cb985c727db9d18fa

Increse max barcode length

view details

solexa

commit sha c19678de09f293b0a989e2f35934329fc79d2db9

Add Novaseq 6K SP configuration

view details

push time in a month

issue commenttree-sitter/tree-sitter

highlight command panics with tree-sitter-cli 0.20.0

I had some difficulty figuring out how to compile and use the tree-sitter cli from source, but I believe that I've managed it and confirmed that it fixes the panic when highlighting my grammar.

ahelwer

comment created time in a month

issue openednextflow-io/nextflow

Nextflow doesn't launch if LAUNCH_FILE contains path with spaces

Bug report

Expected behavior and actual behavior

Expected: Running nextflow should bring up Usage information.

Actual: fails to launch.

Steps to reproduce the problem

I'm not sure of a simple way to reproduce this problem. My launch file contains a path to java that contains a space (see first argument):

❯ cat /Users/nelsonjs/.nextflow/tmp/launcher/nextflow-one_21.04.3/MacBook-Pro.local/classpath-927db714be033db5a75c9f5ad31dc471
"/Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java" "-XX:+TieredCompilation" "-XX:TieredStopAtLevel=1" "-Djava.io.tmpdir=/var/folders/dm/kl5tffts0j51gwljhqfgxx1hcm0144/T/" "-Djava.library.path=/Users/nelsonjs/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java" "-Dfile.encoding=UTF-8" "-Dcapsule.app=nextflow_21.04.3" "-Djava.awt.headless=true" "-Dcapsule.jar=/Users/nelsonjs/.nextflow/framework/21.04.3/nextflow-21.04.3-one.jar" "-classpath" "/Users/nelsonjs/.nextflow/framework/21.04.3/nextflow-21.04.3-one.jar:/Users/nelsonjs/.nextflow/capsule/deps/io/nextflow/nextflow/21.04.3/nextflow-21.04.3.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/apache/ivy/ivy/2.3.0/ivy-2.3.0.jar:/Users/nelsonjs/.nextflow/capsule/deps/io/nextflow/nf-httpfs/21.04.3/nf-httpfs-21.04.3.jar:/Users/nelsonjs/.nextflow/capsule/deps/io/nextflow/nf-commons/21.04.3/nf-commons-21.04.3.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/codehaus/groovy/groovy-nio/3.0.7/groovy-nio-3.0.7.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/codehaus/groovy/groovy-templates/3.0.7/groovy-templates-3.0.7.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/codehaus/groovy/groovy-xml/3.0.7/groovy-xml-3.0.7.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/codehaus/groovy/groovy-json/3.0.7/groovy-json-3.0.7.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/codehaus/groovy/groovy/3.0.7/groovy-3.0.7.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/slf4j/jcl-over-slf4j/1.7.25/jcl-over-slf4j-1.7.25.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/slf4j/jul-to-slf4j/1.7.25/jul-to-slf4j-1.7.25.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/slf4j/log4j-over-slf4j/1.7.25/log4j-over-slf4j-1.7.25.jar:/Users/nelsonjs/.nextflow/capsule/deps/ch/qos/logback/logback-classic/1.1.11/logback-classic-1.1.11.jar:/Users/nelsonjs/.nextflow/capsule/deps/ch/qos/logback/logback-core/1.1.11/logback-core-1.1.11.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/codehaus/gpars/gpars/1.2.1/gpars-1.2.1.jar:/Users/nelsonjs/.nextflow/capsule/deps/ch/grengine/grengine/1.3.0/grengine-1.3.0.jar:/Users/nelsonjs/.nextflow/capsule/deps/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/pf4j/pf4j-update/2.3.0/pf4j-update-2.3.0.jar:/Users/nelsonjs/.nextflow/capsule/deps/commons-codec/commons-codec/1.10/commons-codec-1.10.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/beust/jcommander/1.35/jcommander-1.35.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/esotericsoftware/kryo/kryo/2.24.0/kryo-2.24.0.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/iq80/leveldb/leveldb/0.12/leveldb-0.12.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/eclipse/jgit/org.eclipse.jgit/5.2.1.201812262042-r/org.eclipse.jgit-5.2.1.201812262042-r.jar:/Users/nelsonjs/.nextflow/capsule/deps/javax/mail/mail/1.4.7/mail-1.4.7.jar:/Users/nelsonjs/.nextflow/capsule/deps/javax/activation/activation/1.1.1/activation-1.1.1.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/yaml/snakeyaml/1.18/snakeyaml-1.18.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/jsoup/jsoup/1.11.2/jsoup-1.11.2.jar:/Users/nelsonjs/.nextflow/capsule/deps/jline/jline/2.9/jline-2.9.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/pf4j/pf4j/3.4.1/pf4j-3.4.1.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/google/guava/guava/24.1.1-jre/guava-24.1.1-jre.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/multiverse/multiverse-core/0.7.0/multiverse-core-0.7.0.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/codehaus/jsr166-mirror/jsr166y/1.7.0/jsr166y-1.7.0.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/objenesis/objenesis/2.1/objenesis-2.1.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/iq80/leveldb/leveldb-api/0.12/leveldb-api-0.12.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/jcraft/jsch/0.1.54/jsch-0.1.54.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/jcraft/jzlib/1.1.1/jzlib-1.1.1.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/googlecode/javaewah/JavaEWAH/1.1.6/JavaEWAH-1.1.6.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/github/zafarkhaja/java-semver/0.9.0/java-semver-0.9.0.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/checkerframework/checker-compat-qual/2.0.0/checker-compat-qual-2.0.0.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/google/errorprone/error_prone_annotations/2.1.3/error_prone_annotations-2.1.3.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar:/Users/nelsonjs/.nextflow/capsule/deps/org/codehaus/mojo/animal-sniffer-annotations/1.14/animal-sniffer-annotations-1.14.jar:/Users/nelsonjs/.nextflow/capsule/deps/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar" "nextflow.cli.Launcher"

I found this file by running with NXF_DEBUG=x nextflow and looking for the first line of launcher=.

Program output

$ nextflow
/usr/local/bin/nextflow: line 175: /Library/Internet: No such file or directory

Environment

  • Nextflow version: 21.04.3
  • Java version: 1.8
  • Operating system: macOS
  • Bash version: GNU bash, version 5.1.8(1)-release (x86_64-apple-darwin19.6.0)

Additional context

Suggested fix to nextflow (the bash script at the root of this repo):

425c425
<     declare -a launcher="($(cat "$LAUNCH_FILE"))"
---
>     launcher=($(cat "$LAUNCH_FILE"))

created time in a month

startedms-jpq/sad

started time in a month

startedms-jpq/coq_nvim

started time in a month

startedms-jpq/chadtree

started time in a month

startedms-jpq/gay

started time in a month

pull request commentnf-core/modules

Subworkflow Infrastructure

On sub-dependencies, this seems related to a CS problem, so I'll try to explain how some options that I'm aware of in that field. Much of this will probably be familiar to many readers.

I don't like the fact that you could potentially install exactly the same module multiple times in the same repo but I am struggling to think of a better way that would allow us to create self-enclosed sub-workflows. We definitely can't assume that we can update a module and not have to update any sub-workflows it is used in because e.g. adding an extra input channel will break said sub-workflow - everything needs to be tested and updated together.

Coming from a computer-science perspective, this seems related to the diamond dependency problem. You have two major choices - allow multiple versions of a dependency to exist in a project, or force all to use the same version. It sounds like for nf-core, it's definitely desirable to allow multiple versions, because that means you don't need nearly as much synchronization between different codebases.

NPM, a javascript package manager, historically stores sub-dependencies as nested-directories. Most of the time, this overhead is fine in practice. But because this can suck (duplication of identical code costs time, space, etc), it ships with an npm dedupe command that examines the dependencies and "flattens" the tree where possible. e.g:

workflows/
  libA/
    libB/
      libC/
  libD/
    libC/

becomes

workflows/
  libA/
  libB/
  libC/
  libD/

, with the exception that different versions of packages would still be nested. (e.g: if libB and libD required incompatible versions of libC, libC would stay as two separate subdirectories).

Another approach is to put each version of each dependency in a unique, predictable location, e.g: using the hash as a directory component.

workflows/libA@ab8302dlf/
workflows/libB@c3302da94/
workflows/libC@00f9a9382/
workflows/libC@4938282fa/

However, users probably don't want to type import { libB } from "../workflows/libB@c3302da94/", so this approach usually comes with a smarter "linker" or other system to help map these to human-meaningful names. The package manager pnpm uses an approach like this. (It creates hard-links on the filesystem in the "normal" nested structure to a 'global' store).

However, both of these flattening strategies can come with additional cognitive overhead for the user, and might not be right for nf-core. Editing a dependency during development is usually easier with the more "obvious" nested structure, so that you're free to experiment without affecting other pieces that also use that dependency.

If anyone is interested in a longer read, I'd recommend the article So you want to write a package manager. The core nf-core (hehe) team likely doesn't need to revisit this, but for anyone else interested, it's a good read. Specifically on this issue, it contains the recommendation:

If your language permits it, and the type system won’t choke on it, and the global state risks are negligible, and you’re cool with some binary/process footprint bloating and (probably negligible) runtime performance costs, AND bogging down static analysis and transpilation tooling is OK, then duplication is the shared deps solution you’re looking for. That’s a lot of conditions, but it may still be preferable to reconciliation strategies, as most require user intervention — colloquially known as DEPENDENCY HELL — and all involve potentially uncomfortable compromises in the logic itself.

Emiller88

comment created time in 2 months

push eventStamLab/stampipes

Jemma Nelson

commit sha c4e7f44be379cabd54e332bafa261894dc1977a4

Pin STAR version to 2.4.2a

view details

push time in 2 months

create barnchfwip/miniz_oxide

branch : fix/no_anonymous_params

created branch time in 2 months

fork fwip/miniz_oxide

Rust replacement for miniz

fork in 2 months

push eventStamLab/stampipes

Jemma Nelson

commit sha 2c632f381e8160e84a3aa72be84c887d1e13d2ab

RNA fixups from production

view details

push time in 2 months

issue commenttree-sitter/tree-sitter

highlight command panics with tree-sitter-cli 0.20.0

I am getting the same error with a grammar for awk that I am authoring:

$ npx tree-sitter-cli@0.19.5 generate && npx tree-sitter-cli@0.19.5 highlight test/examples/hello.awk
BEGIN { print "Hello, World" }
$ npx tree-sitter-cli@0.20.0 generate && npx tree-sitter-cli@0.20.0 highlight test/examples/hello.awk
thread 'main' panicked at 'index out of bounds: the len is 2 but the index is 11', cli/src/highlight.rs:359:34
stack backtrace:
   0:        0x1097f6ae4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h5b5b7de1c79b4b45
   1:        0x109819b9e - core::fmt::write::h65ec8c8a6aa7b549
   2:        0x1097ef5fa - std::io::Write::write_fmt::hc52e560ab9e57db4
   3:        0x1097f8b2f - std::panicking::default_hook::{{closure}}::h0974624c30a6e2b5
   4:        0x1097f8639 - std::panicking::default_hook::hda3cfa32e0773aac
   5:        0x1097f9195 - std::panicking::rust_panic_with_hook::h1cdaa4b48d8ea4a1
   6:        0x1097f8cd5 - std::panicking::begin_panic_handler::{{closure}}::h51157b4dbc4c5d91
   7:        0x1097f6f78 - std::sys_common::backtrace::__rust_end_short_backtrace::h8f7fee8b4c5a0c05
   8:        0x1097f8c3a - _rust_begin_unwind
   9:        0x10984bddf - core::panicking::panic_fmt::h28a37220b8fae52a
  10:        0x10984bda6 - core::panicking::panic_bounds_check::h9e722aa90c3cc256
  11:        0x10953eb79 - tree_sitter_cli::highlight::ansi::h3134fb19e99a4e7e
  12:        0x1094fb7f0 - tree_sitter::run::h0d4927a1edc1a911
  13:        0x1094f6c63 - tree_sitter::main::hdb72213bc64bfedf
  14:        0x1094f3c8a - std::sys_common::backtrace::__rust_begin_short_backtrace::h63c96d682da41595
  15:        0x1094f3cac - std::rt::lang_start::{{closure}}::h13a1866a455413a5
  16:        0x1097f9501 - std::rt::lang_start_internal::h37583e1fa93097c4
  17:        0x1094fe249 - _main

Running generate=0.20.0 and highlight=0.19.5 appears to work correctly (produces highlighted code), and the reverse of generate=0.19.5 and highlight=0.20.0 gives a slightly different error (index out of bounds: the len is 2 but the index is 3).

I've uploaded my grammar.js, test/examples/hello.awk, and queries/highlights.scm in this gist: https://gist.github.com/fwip/562ff2ed84de28cc252a0bc457c8d152

Let me know if there's anything else I can do to help debug this issue.

ahelwer

comment created time in 2 months

issue commentfwip/autoswarm

TypeError: Automerge.DocSet is not a constructor

Leaving a note here so I don't forget: https://github.com/automerge/automerge/pull/339 is the new sync protocol to use. :)

draeder

comment created time in 2 months

push eventStamLab/stampipes

Jemma Nelson

commit sha a69a4814f6bfdfbdad9d9fec8b96ada97a3a3469

Add takara UMT trimming script

view details

Jemma Nelson

commit sha fc50802c88d59b11a8b857a147dc5477474a2a20

Add deprecation notice to process_rna_star

view details

Jemma Nelson

commit sha 72e87fc224e0e24c7889e3efb11e69503e18a711

Use Takara-UMT trimming for one library kit In the future we should figure out a better way of organizing this.

view details

Jemma Nelson

commit sha e490ce4c70b8cebb412a1b70e61c75068099bacc

Add LIBRARY_KIT to alignprocess.py

view details

Jemma Nelson

commit sha bbf1228a78a89b0aaa4a9b77f0fd36858c357014

Add generic move_umt_to_tag.py script This functionality is already baked into filter_reads.py, but we want to use it on RNA reads as well. This standalone script will simplify that.

view details

Jemma Nelson

commit sha ca9e3b96461440c6ab8e5d87e2eb0d9a428a1555

Adapter counts & anaquin depend on dup-removed bam

view details

Jemma Nelson

commit sha 88be4ac16d8131f5d18da8fbdab450b26aba400f

Remove dups, don't just mark them.

view details

Jemma Nelson

commit sha c6ff069a17acfdebd21b627cbc22ffe1b6e3f7f1

Add --tagname option to move_umt_to_tag Defaults to "RX", which is the standard tag to put this information in.

view details

Jemma Nelson

commit sha 95be48413fce128db1c6e902f47dce0dff6f7267

Quote library kit method in run.bash

view details

Jemma Nelson

commit sha 41a98c844c5be13011876b06ef19a5ee23c3d0a5

Update logic for Library kit specialization

view details

Jemma Nelson

commit sha 5fc54acba189ac7f61c220d36575d613f1e64232

Move dup removal up top

view details

Jemma Nelson

commit sha 7c1858813bd90c4134f0275a2b64198650ff1f56

Make LIBRARY_KIT info available to aggregations

view details

Jemma Nelson

commit sha db4f0c8944cd34ac5ffd50a4b92de306739d400f

Run on de-duped fastq if UMT present This affects every job that uses Fastq as input: * Kallisto * Kallisto advanced * Ribosomal counts

view details

Jemma Nelson

commit sha f69fad23190c4bf90fd3178af2b4bd9848a607dd

Small UMI fixes

view details

Jemma Nelson

commit sha 1363e0b142a9bbc4d7a03edd3a33e2f46cac77e0

Update library kit method lookup

view details

Jemma Nelson

commit sha 35d2a15d4bbe30000ea75505cb492d1db0f8609e

First draft of RNA alignment in nextflow run.bash script not yet implemented.

view details

Jemma Nelson

commit sha 939ae2af126d6aae98f4f3732e2fc6c3e7810116

first draft of cufflinks agg in Nextflow

view details

Jemma Nelson

commit sha e865ac085439dc2e0713e08d32acd9a7dbcae509

Cleanup alignment script

view details

Jemma Nelson

commit sha 454d186eb6b9fb33ad74e5076de2413d2ee14374

update .gitignore match for temp output

view details

Jemma Nelson

commit sha b834b85edf5d7a320397b6e4d825d46ae9f05e3d

Emit bam index as well on merge

view details

push time in 2 months

PR merged StamLab/stampipes

Feat/takara umt trimming

This adds UMT trimming libraries for Takara v3 RNA libraries.

+1380 -835

0 comment

20 changed files

fwip

pr closed time in 2 months

push eventStamLab/stampipes

Jemma Nelson

commit sha 1d97c4c9fc40d784caa809889ea25c9988cc52e5

Set RNA aggregation version to 1.1

view details

push time in 2 months