profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/jdorn/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

jdorn/json-editor 5590

JSON Schema Based Editor

jdorn/sql-formatter 3882

A lightweight php class for formatting sql statements. Handles automatic indentation and syntax highlighting.

jdorn/php-reports 459

A PHP framework for displaying reports from any data source, including SQL and MongoDB

jdorn/FileSystemCache 48

A simple PHP class for caching data in the filesystem.

jdorn/EditTable 22

A small jQuery plugin to make your HTML tables editable

jdorn/js-form-validator 9

Unobtrusive javascript form validation. Library agnostic and completely customizeable.

jdorn/AssetBundler 6

A PHP class for converting groups of assets into bundles

jdorn/hummingbird 6

Real time web analytics using node.js and web sockets

jdorn/mxn 3

A mapping abstraction library.

jdorn/dorncms-2 2

A complete rewrite of DornCMS

issue openedgrowthbook/growthbook

Detect activation metric bugs

Activation metrics are used when the experiment exposure event is logged before the user actually sees the treatment. These delayed treatments are common sources of bugs - for example, an engineer thinking their code only affects a modal, but it actually changes the underlying page too.

We should be able to detect these bugs by in-essence doing an A/A test of users who fired the exposure event, but did not fire the activation metric.

created time in 2 hours

issue openedgrowthbook/growthbook

Ability to delete an experiment phase

created time in 2 hours

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha d6f4a5b06d4b7e6b9c1028a0f0750f1b7474a731

Fix GitHub Action for releases

view details

push time in 7 hours

release growthbook/growthbook

v0.5.0

released time in 7 hours

created taggrowthbook/growthbook

tagv0.5.0

The Open Source A/B Testing Platform

created time in 7 hours

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 358f8fa5bf2911af8347e623e5afdf2cc6c4a32d

Fix notebook error when using old metrics. Add config.yml docs for notebookRunQuery

view details

push time in 7 hours

delete branch growthbook/growthbook

delete branch : notebook

delete time in 18 hours

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 27d5413bc8ead08a59e9c1acf57105648b064fca

Generate Jupyter notebook of experiment results (#81) Also: * Publish stats engine as a standalone python package * Store raw query result rows in addition to the processed results * Switch back-end to use new `gbstats` library for analysis, show raw query results as rows. * Switch docker to use `python-slim` base image instead of `node-alpine` * Downgrade typescript to 4.3.5 to fix ts-server bugs * Display query results as a table instead of JSON string

view details

push time in 18 hours

PR merged growthbook/growthbook

Generate Jupyter notebook of experiment results

Fixes #80

TODO:

  • [x] Type definition for the Notebook file format
  • [x] Code to generate skeleton notebook file (no runQuery, stats, or results visualization)
  • [x] Add runQuery setting for data sources
  • [x] UI to set the runQuery code
  • [x] Store raw db rows in Query documents
  • [x] Add runQuery logic to generated notebook
  • [x] Add results table to generated notebook
  • [x] UI and API endpoint to download a notebook from the results page
  • [x] Move query result post-processing code (and tests) to Python
  • [x] Move stddev correction code (and tests) to Python
  • [x] Move SRM check code (and tests) to Python
  • [x] Move variation id mismatch check code (and tests) to Python
  • [x] Move stats post-processing code (and tests) to Python
  • [x] Make stats engine a standalone Python package available on pypi
  • [x] Add data quality checks to generated notebook
  • [x] Add stats, and pre/post-processing code to generated notebook
  • [x] Add docs
  • [x] Add Jupyter Notebook feature to main README
  • [x] Add python tests
  • [x] Make the export button look better (icon, etc.)
  • [x] Make back-end use gbstats package for normal stats computation
  • [x] Publish gbstats package on pypi
  • [x] Test docker image
+2990 -573

1 comment

62 changed files

jdorn

pr closed time in 18 hours

issue closedgrowthbook/growthbook

Generate Jupyter Notebook from Results

We currently allow viewing the raw SQL queries on experiment results. We should also be able to generate a Jupyter Notebook on the fly that recreates the experiment results. This will let data teams easily do further analysis on an experiment when required.

For configuration, each data source will need to define a Python code cell with a runQuery function. For example:

def runQuery(sql):
  # TODO: connect to database, run the query, return list of rows
  return [];

Then we can generate a notebook with the following cells

  1. Markdown field with the experiment name, hypothesis, link to GrowthBook results, etc.
  2. Python imports (numpy, scipy, the GrowthBook stats engine, and a plotting library if needed)
  3. The runQuery definition for the datasource
  4. Run SQL queries (number of users plus one for each metric)
  5. Clean up the raw SQL rows and get the data ready for the stats engine
  6. Call the stats engine for each metric
  7. Post-process the stats results
  8. Display any warnings (e.g. SRM)
  9. Render the results output as a table

closed time in 18 hours

jdorn

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 1a99aec2ca43132a0c10e574acca004b8d27bebc

Fix docker warnings

view details

push time in 19 hours

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha d09a2202ccbda3bca8ebb62a43e201a713904c80

Switch back-end to use gbstats library for analysis, show raw query results as rows.

view details

push time in a day

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 48cc33daac4329d3de3f6316becf81d4a5623b88

Move notebook generation to python

view details

push time in a day

startednikolaik/docker-python-nodejs

started time in a day

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 259744b3dbb420b23eeb967c57b9ccec19c955d8

Fix notebook bugs

view details

push time in 2 days

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha c071b5a458326eae57a434ed81de8b3df56138aa

Add tests for stats package and add to CI pipeline

view details

push time in 3 days

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 122c41a4760870f2205bc15f6d8b5f9137071d39

Add documentation for Jupyter notebook export

view details

push time in 3 days

delete branch growthbook/growthbook

delete branch : import-running

delete time in 3 days

PR merged growthbook/growthbook

Fixes for importing a running experiment.
  • [x] While importing, if there was experiment data within the past 3 days, default to setting the status to "running"
  • [x] Allow changing the status within the import modal
  • [x] When importing a running experiment, don't ask for an End Date
  • [x] Fix broken css on "Stop Experiment" modal
+27 -10

1 comment

3 changed files

jdorn

pr closed time in 3 days

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 861aab1b2f278a5e2cd9e7397716b64eca43ca27

Fixes for importing a running experiment. (#82)

view details

push time in 3 days

PR opened growthbook/growthbook

Fixes for importing a running experiment.
  • [x] While importing, if there was experiment data within the past 3 days, default to setting the status to "running"
  • [x] Allow changing the status within the import modal
  • [x] When importing a running experiment, don't ask for an End Date
  • [x] Fix broken css on "Stop Experiment" modal
+27 -10

0 comment

3 changed files

pr created time in 3 days

create barnchgrowthbook/growthbook

branch : import-running

created branch time in 3 days

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 131f45492f10bca176c85b5b83a9f9535234d491

Add SRM check to notebook, add example `runQuery` method

view details

push time in 3 days

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 096c187c75bf496b462093acfcb3d14a1cdfb2aa

Make the notebook actually run! (also downgrade typescript to 4.3.5)

view details

push time in 4 days

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha b28cb23df307d19dde716a5369bfafa3578990ff

Export Jupyter notebook

view details

push time in 5 days

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 0b3a6c01ee0d85da270bd23556858be8064c05d1

Store raw query results

view details

push time in 5 days

PR opened growthbook/growthbook

Generate Jupyter notebook of experiment results

Fixes #80

TODO:

  • [x] Type definition for the Notebook file format
  • [x] Add runQuery setting for data sources
  • [ ] UI to set the runQuery code
  • [ ] Store raw db rows in Query documents
  • [ ] Code to generate skeleton notebook file with placeholders for the stats and results table
  • [ ] UI and API endpoint to download a notebook from the results page
  • [ ] Make stats engine a standalone Python package available on pypi
  • [ ] Move stddev correction code to Python
  • [ ] Move SRM check code to Python
  • [ ] Move stats post-processing code to Python
  • [ ] Add stats to generated notebook
  • [ ] Add results table to generated notebook
+317 -0

0 comment

4 changed files

pr created time in 6 days

create barnchgrowthbook/growthbook

branch : notebook

created branch time in 6 days

push eventgrowthbook/growthbook

Jeremy Dorn

commit sha 4a21ed90eb2618ff9d79f6833ee269c73f017674

View and edit the project for an experiment, default to the current project when creating an experiment, set metrics, dimensions, and segments for a project.

view details

push time in 6 days

issue openedgrowthbook/growthbook

Generate Jupyter Notebook from Results

We currently allow viewing the raw SQL queries on experiment results. We should also be able to generate a Jupyter Notebook on the fly that recreates the experiment results. This will let data teams easily do further analysis on an experiment when required.

For configuration, each data source will need to define a Python code cell with a runQuery function. For example:

def runQuery(sql):
  # TODO: connect to database, run the query, return list of rows
  return [];

Then we can generate a notebook with the following cells

  1. Python imports (numpy, scipy, the GrowthBook stats engine, and a plotting library)
  2. The runQuery definition for the datasource
  3. Run SQL queries (users plus one for each metric)
  4. Clean up the raw SQL rows and get the data ready for the stats engine
  5. Call the stats engine for each metric
  6. Post-process the stats results
  7. Render the output as a table

created time in 6 days