profile
viewpoint
Tim Wisniewski timwis London https://timwis.com Civic technologist specialising in navigating bureaucracy & culture change. Former Chief Data Officer @PhiladelphiaGov. Just moved to London!

choojs/choo 6451

:steam_locomotive::train: - sturdy 4kb frontend framework

choojs/bankai 1054

:station: - friendly web compiler

tbranyen/hyperlist 284

A performant virtual scrolling list utility capable of rendering millions of rows

choojs/choo-handbook 264

🚂✋📖 - Learn the choo framework through a set of exercises

choojs/nanotiming 32

⏲ - Small timing library

CodeForPhilly/stately 17

Form-driven workflow engine

DataSF/open-data-explorer 12

To promote exploration and use of open data - currently in beta

CodeForPhilly/philly-ward-leaders 11

The players behind elections

timwis/actionable 7

A GTD-inspired todo list that lets you focus on immediately actionable tasks (tasks with no dependencies).

mdb/node-phli 6

A Node.js module for interacting with the City of Philadelphia's Licenses & Inspections data.

issue commentFlyOrBoom/mercator

SVG filters not working on firefox

Just to reiterate though, it works fine on your demo site

timwis

comment created time in 6 days

startedleits/MeetingBar

started time in 8 days

startedmengshukeji/Luckysheet

started time in 8 days

issue openedFlyOrBoom/mercator

Not working on firefox

Hey, love the idea for this! It doesn't seem to be working on Firefox for me... I'm using greasemonkey, and it shows the Studio Mini controls, but nothing happens when I adjust them while on google meet. It works fine on your demo site though, which suggests it may be an issue with the greasemonkey script..maybe an old version?

created time in 8 days

startedFlyOrBoom/mercator

started time in 10 days

startedrnkn/olivetti

started time in 18 days

startedjoostkremers/writeroom-mode

started time in 18 days

startedShabang-Systems/Condution

started time in 18 days

startedactiverecord-hackery/ransack

started time in 19 days

startedheartcombo/simple_form

started time in 19 days

issue commentyakyak/yakyak

Frequently stops sending messages (disconnects?)

What do you mean open yakyak on a command line? Do you mean check the javascript console in the developer tools of the electron application? If so, I have and there's no logs or network activity.

timwis

comment created time in 21 days

issue openedyakyak/yakyak

Frequently stops sending messages (disconnects?)

Hi! Big fan of YakYak, but I've noticed that once or twice per day my messages stop sending. They appear faded, as if they're trying to send, but never do. I fix it by quitting and restarting the application (at which point the unsent messages are gone).

Any idea what might be causing this? For what it's worth I'm on an LTE connection (home LTE router) but that's never caused an issue with any other application (whether netflix or SSH or anything else).

created time in 23 days

startedTeamTopologies/Team-Dependencies-Tracking

started time in a month

issue commentJarvusInnovations/gitsheets

Reorganize repository

Love it

themightychris

comment created time in a month

startedfoambubble/foam

started time in a month

startedmaxgoedjen/secretive

started time in a month

startedmickael-kerjean/filestash

started time in a month

startedjshjohnson/Choices

started time in a month

startedbasecamp/geared_pagination

started time in a month

startedjeapostrophe/exp

started time in a month

startedformtastic/formtastic

started time in 2 months

startedbrentonk/adaptive-wrap-vp

started time in 2 months

startedplatanus/activeadmin_addons

started time in 2 months

startedbrainspec/enumerize

started time in 2 months

startedadzap/validates_timeliness

started time in 2 months

startedjohang/btfs

started time in 2 months

startedrxhanson/Rectangle

started time in 2 months

startedxgenecloud/xgenecloud

started time in 2 months

issue commentgera2ld/markmap-lib

Non-linear references

How about a faded dashed line connecting them that, when you hover on it, increases its opacity and emphasises the nodes it connects?

timwis

comment created time in 2 months

issue commentgera2ld/markmap-lib

Non-linear references

Do you mean aesthetically? I’d assume it’s just another line between the two nodes. Or do you mean how to identify which node to connect to without IDs?

timwis

comment created time in 2 months

issue closedmozilla/contain-facebook

Toolbar button missing

<!-- Feel free to ignore this Issue template if you just want to ask or suggest something. If you experience an Issue then please provide all asked information.

Also please make sure that:

  • "Firefox will: Never remember history" in the Firefox Preferences/Options under "Privacy & Security > History" is NOT selected
  • You are NOT using Firefox in a Private Window
  • You can see a grayed out but ticked Checkbox with the description "Enable Container Tabs" in the Firefox Preferences/Options under "Tabs" -->
  • Facebook Container Version: 2.1.1
  • Operating System + Version: macOS Catalina 10.15.5
  • Firefox Version: 77.0.1
  • Other installed Add-ons + Version + Enabled/Disabled-Status: <!-- To be able to copy & paste the full list of your Add-ons navigate to "about:support" and scroll down to "Extensions" -->
    • 1Password extension (desktop app required)
    • Amazon.co.uk
    • Bing
    • Bypass Paywalls Clean
    • Chambers (UK)
    • Cookie Quick Manager
    • DuckDuckGo
    • eBay
    • Facebook Container
    • Firefox Multi-Account Containers
    • Google
    • Google Container
    • I don't care about cookies
    • Markdown Here
    • Metastream Remote
    • Octotree
    • Refined GitHub
    • Stylus
    • Twitter
    • uBlock Origin
    • Vimium
    • Vue.js devtools
    • Wikipedia (en)
    • Pinboard
    • Tabliss
    • WAVE Evaluation Tool
    • ZenHub for GitHub

Actual behavior

The add-ons toolbar does not have an icon for this extension. Screenshot 2020-06-12 at 13 58 19

Expected behavior

The toolbar should have an icon so I can whitelist other pages, as per the docs.

Steps to reproduce

Notes

closed time in 2 months

timwis

issue commentmozilla/contain-facebook

Toolbar button missing

Solution: In the main firefox menu, click "Customise" and look for the facebook container icon (a fence), then drag it to the add-ons toolbar.

timwis

comment created time in 2 months

issue openedmozilla/contain-facebook

Toolbar button missing

<!-- Feel free to ignore this Issue template if you just want to ask or suggest something. If you experience an Issue then please provide all asked information.

Also please make sure that:

  • "Firefox will: Never remember history" in the Firefox Preferences/Options under "Privacy & Security > History" is NOT selected
  • You are NOT using Firefox in a Private Window
  • You can see a grayed out but ticked Checkbox with the description "Enable Container Tabs" in the Firefox Preferences/Options under "Tabs" -->
  • Facebook Container Version: 2.1.1
  • Operating System + Version: macOS Catalina 10.15.5
  • Firefox Version: 77.0.1
  • Other installed Add-ons + Version + Enabled/Disabled-Status: <!-- To be able to copy & paste the full list of your Add-ons navigate to "about:support" and scroll down to "Extensions" -->
    • 1Password extension (desktop app required)
    • Amazon.co.uk
    • Bing
    • Bypass Paywalls Clean
    • Chambers (UK)
    • Cookie Quick Manager
    • DuckDuckGo
    • eBay
    • Facebook Container
    • Firefox Multi-Account Containers
    • Google
    • Google Container
    • I don't care about cookies
    • Markdown Here
    • Metastream Remote
    • Octotree
    • Refined GitHub
    • Stylus
    • Twitter
    • uBlock Origin
    • Vimium
    • Vue.js devtools
    • Wikipedia (en)
    • Pinboard
    • Tabliss
    • WAVE Evaluation Tool
    • ZenHub for GitHub

Actual behavior

The add-ons toolbar does not have an icon for this extension. Screenshot 2020-06-12 at 13 58 19

Expected behavior

The toolbar should have an icon so I can whitelist other pages, as per the docs.

Steps to reproduce

Notes

created time in 2 months

startedyakyak/yakyak

started time in 2 months

startedoutline/rich-markdown-editor

started time in 2 months

starteddonrestarone/cookie-authentication-backend

started time in 2 months

startedchriskacerguis/codeigniter-restserver

started time in 2 months

startedsickcodes/Docker-OSX

started time in 2 months

PR opened Police-Data-Accessibility-Project/Police-Data-Accessibility-Project

Update description

Just updating the readme with the latest description from this google doc and tidying up the markdown a bit.

I'm sure within a matter of hours or days this will be updated much more comprehensively — this is just an "in the meantime" thing since this repo is being linked to a lot.

Note that this removes the "It is my belief" and "big undertaking" lines (replacing them with the latest description) but leaves in the goals bit (those didn't have a direct replacement from the doc).

Hope this helps!

+18 -13

0 comment

1 changed file

pr created time in 2 months

push eventtimwis/Police-Data-Accessibility-Project

Tim Wisniewski

commit sha ac9e8a3fa75959d5adbef388ea1339fbfb59a49b

Update description - Uses latest description from https://docs.google.com/document/d/14Yg7e73VxXjzpJEwtK-EFkxSdQGdSeGxA4D1NMjKSYw/edit#heading=h.fearqwdeyjim - Tidies up the markdown a bit

view details

push time in 2 months

startedricardochimal/taps

started time in 2 months

startedjoelparkerhenderson/architecture_decision_record

started time in 2 months

issue openedgera2ld/markmap-lib

Non-linear references

Hi! Really cool tool. Is there any support for non-liner references? For example when a grandchild of A has a connection to a child of B, or an item is connected to multiple parents? Basically, drawing a connection between two nodes that are not in the same branch.

# Goals

## Open new branch
- Close property deal
- Hire additional team members [connects to Secure new investment]

## Secure new investment
- Prepare business plan
- Find investors

created time in 2 months

startedgera2ld/markmap-lib

started time in 2 months

startedwearebraid/vue-formulate

started time in 2 months

starteddavestewart/vue-class-store

started time in 2 months

startedFinancial-Times/engineering-progression

started time in 3 months

startedrethab/contributify

started time in 3 months

starteddohliam/dropin-minimal-css

started time in 3 months

startedxz/fonts

started time in 3 months

startedxz/new.css

started time in 3 months

startedvlado/activerecord-cte

started time in 3 months

startedtwalpole/apparition

started time in 3 months

startedrubycdp/cuprite

started time in 3 months

startedClosureTree/closure_tree

started time in 3 months

issue openedtimwis/importui

Coerce types

CSV columns are all strings by default. We'll need to attempt to coerce them into the desired type of the field they're matched to. ajv does this during validation.

created time in 3 months

issue openedcaolan/highland

Returning an async iterable

Hi! Is it possible to get an async iterable from a highland object? I'd like to be able to consume it with for-await-of if possible.

created time in 3 months

issue openedtimwis/importui

Import function should return an async iterable

Instead of an uploadRow function and an uploadAll function, the import/upload function should just return an async iterable, so the consumer syntax would look like:

const rows = await import({ file, header, delay })
for await (const row of rows) {
  await axios.post('/users', row)
}

or to upload all at once:

const rows = await import({ file, header, delay })
await axios.post('/users/import', [...rows])

created time in 3 months

push eventtimwis/importui

timwis

commit sha 55af72ea1c820aae50b7b9101055dcbf379f2eae

Move ImportUI to its own component

view details

push time in 3 months

push eventtimwis/importui

timwis

commit sha 5effe16a74611a2abbeea90d062101619f8843b4

Use highland.js for streams

view details

push time in 3 months

issue commentJarvusInnovations/gitsheets

Discussion: Improve consistency of stream interface

Hm, I forget.. will JavaScript await the argument before it runs the surrounding pipeline function in that example? If not, it would be passing pipeline an argument of type Promise. Anyway if that were the case, we could just do const writeStream = await gitSheets.import() before calling pipeline. You're right.

What would be even better is if pipeline simply worked with async/promisy arguments, and it just may, particularly if it's an async generator / async iterable.

timwis

comment created time in 3 months

issue commentJarvusInnovations/gitsheets

Discussion: Improve consistency of stream interface

I've been knee-deep in stream land for the past few days and finally remember enough to be able to contribute to this conversation again.

What you've proposed may work, but (editing this part).

I've been considering a few alternatives.

A. Use Node's pipeline utility instead of .pipe.

In this case, server.js and cli.js would do something like this:

const treeHash = await pipeline(
  payload,
  csvParser({ strict: true }),
  gitSheets.import({
    parentRef: ref,
    saveToBranch: branch
  })
)

This would require gitSheets.import() synchronously create/return a Writable stream. The tricky part with that is we need to do some async things before we can start writing (get the config and create a tree). We may be able to do that in some sort of init hook of the Writable stream, and have it not enter a 'ready' state until that's done.

You'll note that pipeline is a promise which returns the resolved value of the promise returned by the sink/destination.

This looks pretty nice, and cleanly solves the problem of keeping csv parsing/formatting in the same place for both import and export. But what about the consistency of the compare and merge methods? Should they return streams too, just to be consistent?

B. Async iterables

These are a relatively new feature for JavaScript, though the seeds have been around since 2015 with generators. Any object can implement the 'iterable' interface by adding a property with the key Symbol.iterator and the value being a method that returns { value: any, done: boolean }. More recently, the 'async iterable' interface was added, with the key Symbol.asyncIterator. And the 'for-await-of' syntax was added, allowing easy iteration. Streams in Node v10+ are automatically async iterables. Strings and arrays are also iterables. I feel like I've got my head 85% wrapped around this, but it seems to address a lot of the issues with mixing promises and streams. It's annoying to deal with interop but that's probably just because (a) I'm still not 100% around it, and (b) there's not a ton of great toolkits out there for it yet.

  async _writeDataToTree ({ data, treeObject, pathTemplate }) {
    const pendingWrites = [];

    for await (const row of data) {
      let path
      let contents

      try {
        path = this._renderTemplate(pathTemplate, row);
        contents = this._serialize(row);
      } catch (err) {
        throw new SerializationError(err.message);
      }

      pendingWrites.push(treeObject.writeChild(`${path}.toml`, contents));
    }

    await Promise.all(pendingWrites)
    return treeObject.write()
  }

Using a lib like streaming-iterables (below), this could be simplified further.

  async _writeDataToTree ({ data, treeObject, pathTemplate }) {
    function writeRow (row) {
      const path = this._renderTemplate(pathTemplate, row);
      const contents = this._serialize(row);
      return treeObject.writeChild(`${path}.toml`, contents);
    }

    await pipeline(
      () => data,
      map(writeRow) // could also use parallelMap
    )
    return treeObject.write()
  }

C. Helper libraries

I've come across a few stream toolkits that make this all look much nicer and even improves the semantics of mixing streams with promises, though they all tend to carry some of their own complexity around inter-op. I may just not have cracked it yet.

timwis

comment created time in 3 months

startedcaolan/highland

started time in 3 months

issue openedhughsk/from2

Support async iteration

Hello! I've noticed that streams created with this library do not support async iteration. This appears to be because it depends on readable-stream v2.x.x, and Symbol.asyncIterator was added to that library in v3.x.x, when it started supporting Node v10. I noticed in #21 that there was reluctance to upgrade to readable-stream v3.x.x, but that was more than a year and a half ago.

Would you be open to a PR that updates the dependency? Or does that issue still remain? Alternatively, any suggestions for how I might support async iteration in streams created by this library?

created time in 3 months

issue openedtimwis/importui

Support bulk upload

At the moment, uploading involves sending each request individually, assuming most APIs just have a simple POST /objects resource. But we should also support APIs that have a bulk creation resource, where we stream one large JSON payload. Or line-delimited JSON. And perhaps we allow implementers to configure the batch size.

created time in 3 months

issue openedtimwis/importui

Support nested payload formats

Many APIs require the payload be nested in a {"data": {...}} property rather than all top-level. Some others may require some more complex nesting that is difficult to represent in a CSV file.

The best solution is probably to offer implementers a way to write a "transform" (map) function in the config file that's applied to each row.

A simple one (and maybe we offer this one as some sort of preset) would be:

(row) => ({ data: row })

A more complex one would be an example where the API allows many phone numbers to be passed as an array, but the CSV file simply has a phone1 and phon2 column:

(row) => {
  const { phone1, phone2, ...rest } = row
  return {
    ...rest,
    phones: [ phone1, phone2 ]
  }
}

created time in 3 months

issue openedtimwis/importui

Validate each row

Use ajv (or similar) to validate each row against json schema before sending.

But what to do when a row fails validation? Ideally, uploading would pause and the UI would offer a form to fix the row manually and resume uploading.

created time in 3 months

startednikersify/pico

started time in 3 months

startedalkihis/line-file-reader

started time in 3 months

push eventtimwis/importui

timwis

commit sha 01ff49de9b428c0fed62230de1d6bfb012026ceb

Fix missing setimmediate bug This was not an issue before I deleted node_modules and reinstalled 🤔

view details

timwis

commit sha b365060151f38a6464f07a8f980f0f8bd0372e86

Show ignored columns with grey backgrounds

view details

push time in 3 months

create barnchtimwis/importui

branch : master

created branch time in 3 months

created repositorytimwis/importui

created time in 3 months

startednock/nock

started time in 3 months

startednode-file-api/file-api

started time in 3 months

issue commentmafintosh/csv-parser

Example of browser usage?

I was able to get it to work using filereader-stream.

import fileReaderStream from "filereader-stream";
import csv from "csv-parser";

const fileEl = document.getElementById("file");

fileEl.addEventListener("change", event => {
  const files = event.target.files;
  if (files.length === 0) return;

  fileReaderStream(files[0])
    .pipe(csv())
    .on("data", data => console.log(data));
});

I'm not sure how to verify that it's not simply loading the whole file into memory anyway though :/

timwis

comment created time in 3 months

startedDamonOehlman/filestream

started time in 3 months

startedmaxogden/filereader-stream

started time in 3 months

pull request commentmikker/passwordless

Store hashed tokens

Okay, I think I've incorporated all of your feedback. I dropped the separate TokenGenerator module in order to keep token generation and hashing functionality separate. I moved hashing to a static Passwordless.digest method as you suggested, though the only way I could see to do that was on the Passwordless module - let me know if there's a better way. I also added a couple tests for the configurable bits.

Beyond this, I think it's just a matter of updating the readme to reflect (1) how to upgrade/migrate, and (2) how to customise hashing.

Looking forward to your feedback.

timwis

comment created time in 3 months

Pull request review commentmikker/passwordless

Store hashed tokens (WIP)

 module Passwordless   mattr_accessor(:default_from_address) { "CHANGE_ME@example.com" }   mattr_accessor(:token_generator) { UrlSafeBase64Generator.new }+  mattr_accessor(:digest_algorithm) { "SHA256" }+  mattr_accessor(:digest_secret) { lambda { Rails.application.secret_key_base } }

According to the docs this should get it in all 3 environments. Note that deivse uses a helper class for this to support multiple versions of rails out of the box. We could copy that, or just point them here if they're using an older rails version.

timwis

comment created time in 3 months

Pull request review commentmikker/passwordless

Store hashed tokens (WIP)

 def create_session(attrs = {})       refute_nil session.expires_at       refute_nil session.timeout_at       refute_nil session.token+      refute_nil session.token_digest+    end++    test "it generates a digest" do+      session = Session.new+      session.validate++      assert_equal 64, session.token_digest.length+      refute_equal session.token, session.token_digest+    end++    test "with a custom digest algorithm" do+      old_digest_algorithm = Passwordless.digest_algorithm++      Passwordless.digest_algorithm = "SHA1"++      session = Session.new+      session.validate++      assert_equal 40, session.token_digest.length++      Passwordless.digest_algorithm = old_digest_algorithm+    end++    test "with a custom digest secret" do

This test doesn't make as much sense in the session model test, but I wasn't sure where else to put it, or how to test this feature via the session model.

timwis

comment created time in 3 months

push eventtimwis/passwordless

timwis

commit sha b1e9ac33e603b859261ba6505ff4ccffa9fe68d9

Use static digest method in Passwordless module

view details

push time in 3 months

pull request commentCityOfPhiladelphia/vizwit-pages

Update li-clean-seal.html

Great! Happy to help :)

Kistine

comment created time in 3 months

startedtannercollin/standardnotes-fs

started time in 3 months

pull request commentCityOfPhiladelphia/vizwit-pages

Update li-clean-seal.html

btw, if you go into the browser's developer tools when you load the vizwit page, then click on the network tab, you can see the failed request (it's red). If you go to the 'response' subtab, it will show you the error from carto:

invalid input syntax for type timestamp: \"\"

Screenshot 2020-05-06 at 13 46 19

If you look at the details of the request (in the headers or params subtab), it will give you the URL, which contains the SQL that vizwit actually generated:

SELECT count(*) as value,
       date_trunc('month', workordercompleteddate) as label 
FROM clean_seal
WHERE (workordercompleteddate != '')
GROUP BY date_trunc('month', workordercompleteddate)
ORDER BY label 

Judging by the error, it looks like the issue is that the WHERE clause is comparing a timestamp field to a string, which must be invalid for some reason 🤔 But IS NOT NULL seems to work.

Kistine

comment created time in 3 months

pull request commentCityOfPhiladelphia/vizwit-pages

Update li-clean-seal.html

Hm, when I remove the 'where' entirely, I'm not getting any blank values; just nulls. To filter out nulls, you'd do:

          "type": "is not",
            "value": null
         }

Did you try that?

Kistine

comment created time in 3 months

more