profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/jermnelson/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Jeremy Nelson jermnelson Stanford University Libraries Colorado Springs, Colorado

jermnelson/BIBFRAME-Datastore 13

BIBFRAME Datastore is a Linked-Data project for managing bibliographic records and operational data focused on libraries and other similar information-based organizations.

jermnelson/aristotle-library-apps 12

Aristotle Library Apps are HTML5 Django mobile and tablet apps for discovering, accessing, and managing library information. Bibliographic records along with other library and organizational specific informations are linked, stored, and managed using Redis.

jermnelson/bibframe-catalog 7

Web-based search and display catalog for BIBFRAME linked-data.

jermnelson/bibmorph 3

A command-line and Haskell Library for converting bibliographic records and metadata entities between different vocabularies. This initial version is focused on MARC21 records to BIBFRAME RDF/XML with other conversions being planned.

jermnelson/ala2013-loc-presentation 1

Presentation for Library of Congress BIBFRAME session at ALA 2013 in Chicago, IL.

jermnelson/automatic-bibframe-classification 1

Source code and wiki supporting draft of an article on using machine learning techniques for classifying MARC21 records into BIBFRAME Creative Works for the Redis Library Services Platform

jermnelson/bibframe-socket 1

A lightweight jython Socket server that listens and processes incoming MARC XML into BIBFRAME rdf/xml using Saxon XQuery and https://github.com/lcnetdev/marc2bibframe

jermnelson/bibframe-solr 1

BIBFRAME-based Solr Configuration for use by the Aristotle Library Apps and the BIBFRAME-datastore projects

AI4LAM/metadata-wg-notebooks 0

A collection of Jupyter notebooks used for interactive learning sessions in the AI4LAM Metadata working group meetings.

jermnelson/adr-cc-utilities 0

Flask application tha supports common and helpful task useful for managing a Fedora Commons digital repository. Documentation available at https://readthedocs.org/projects/adr-cc-utilities-web-app

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha 94918fa49090695dba3cdbe57e690b663595c274

Running just test workflow

view details

push time in 5 hours

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha 7654e6451383bb7891953574792507882d72f7d2

Running just test workflow

view details

push time in 5 hours

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha 8f0c5a1b44d63c0e156af589c784b06f2b3a4c31

Running just test workflow

view details

push time in 5 hours

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha bbd074c1f93fa6b717b55e69f2754ee27105100b

Running just test workflow

view details

push time in 6 hours

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha 435a3b1a75af681de3f1c7a9e7f070197efb71d8

Running just test workflow

view details

push time in 6 hours

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha 2325021e510075dcb82b71e78e572040ebef7965

New circle-ci configuration

view details

push time in 6 hours

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha f972b96a123f15bd6fab02bdb3daf085ff099ec7

New circle-ci configuration

view details

push time in 6 hours

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha fa314e6cffd7616f94f82a606e1e0ea32209ee00

New circle-ci configuration

view details

push time in 6 hours

create barnchsul-dlss/dlme-airflow

branch : t11-circle-ci

created branch time in 6 hours

delete branch sul-dlss/dlme-airflow

delete branch : t9-add-flake8

delete time in 13 hours

delete branch sul-dlss/dlme-airflow

delete branch : t13-dev-readme

delete time in 13 hours

delete branch sul-dlss/dlme-airflow

delete branch : t10-testing

delete time in 13 hours

delete branch sul-dlss/dlme-airflow

delete branch : add-git-to-docker-image

delete time in 13 hours

push eventsul-dlss/dlme-airflow

Aaron Collier

commit sha cc9b329e9dbf802a645ca531c0f4d090720ecc20

Add git to the docker image so it is availble to the BashOperator

view details

Jeremy Nelson

commit sha 069743c17bdcac1f0d48d8d651f29f049c921817

Merge pull request #16 from sul-dlss/add-git-to-docker-image Add git to the docker image so it is available to the BashOperator

view details

push time in 13 hours

PR merged sul-dlss/dlme-airflow

Add git to the docker image so it is available to the BashOperator

This install git into the docker image so that it is available within Airflow and BashOperators. This will be necessary for cloning metadata and pushing changes.

+4 -0

0 comment

1 changed file

aaron-collier

pr closed time in 13 hours

PullRequestReviewEvent

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha 384b3777b3c9a97735fddee6026b6e8daa2a0457

Simplify test

view details

push time in a day

PR opened sul-dlss/dlme-airflow

Adds pytest with example test

Fixes #10

Created an example unit test for the copydir module using pytest mocking and using a temporary path.

Updated README with directions on using pytest.

+30 -0

0 comment

3 changed files

pr created time in a day

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha f8de6e73734a5ff16b217ca0c2660e3a77c6b3ee

Tests copydir module

view details

Jeremy Nelson

commit sha 83189ff5e8e4ff46a0d2a90621a3e593baa072e3

Documentation on running tests

view details

push time in a day

create barnchsul-dlss/dlme-airflow

branch : t10-testing

created branch time in a day

PR opened sul-dlss/dlme-airflow

README Dev updates.

Fixes #13

Directions for creating a virtual environment, installing dependencies, and then running flake8 and black to README.

+20 -1

0 comment

1 changed file

pr created time in a day

create barnchsul-dlss/dlme-airflow

branch : t13-dev-readme

created branch time in a day

PR opened sul-dlss/dlme-airflow

Adds flake8 and black

Fixes #9

Adds flake8 linter to project. Also adds black code formatter to make it easier to pass the linter!

+51 -48

0 comment

7 changed files

pr created time in a day

push eventsul-dlss/dlme-airflow

Jeremy Nelson

commit sha 280ee1af194ef609364934f912a0bf65fa06476a

Adds setup.cfg for flake8 to support black

view details

push time in a day

create barnchsul-dlss/dlme-airflow

branch : t9-add-flake8

created branch time in a day

issue commentsul-dlss/dlme-airflow

Add python linter

Ad hoc assigning myself to this ticket but unable to assign using Github (maybe related to not having write access?)

aaron-collier

comment created time in a day

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentsul-dlss/dlme-airflow

Initial docker infrastructure

+from datetime import timedelta+from textwrap import dedent++# The DAG object; we'll need this to instantiate a DAG+from airflow import DAG++# Operators; we need this to operate!+from airflow.operators.bash import BashOperator+from airflow.operators.python import PythonOperator+from airflow.utils.dates import days_ago++import sys

I think in a follow-up PR we could either do one of the follow:

  • In the Dockerfile add /opt/dlme_airflow/ to the PYTHONPATH variable
  • Allow dlme_airflow to be installed as a python package using a binary wheel distribution with pip, see documentation
aaron-collier

comment created time in 2 days

Pull request review commentsul-dlss/dlme-airflow

Initial docker infrastructure

 # dlme-airflow This is a new repository to capture the work related to the DLME ETL Pipeline and establish airflow++NOTE: This is a work-in-progress++# Getting Started++## Initialize local Docker infrastructure++```+docker compose up airflow-init+```++## Start local Docker Resources++```+docker compose up+```++## Adding new DAGs to the image++After creating a new DAG or editing an existing DAG, the resources must be restarted, and in the case of dlme-airflow, the base image rebuilt.++```+docker build . -f Dockerfile --tag suldlss/dlme-airflow:latest+```++then stop and restart all of the docker compose resources.++## Visit the Airflow dashboard++open your browser to `http://localhost:8080`++## Enable the DAGs you wish to run locallay++## Run inidividual DAGs

typo: should be individual

aaron-collier

comment created time in 2 days