profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/yuanbit/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

yuanbit/FinBERT-QA 26

Financial Domain Question Answering with pre-trained BERT Language Model

yuanbit/FinBERT-QA-notebooks 7

Notebooks for fine-tuning a BERT model and training a LSTM model for financial QA

yuanbit/sentiment140-biLSTM 2

Sentiment Analysis of Tweets using biLSTM built with PyTorch

yuanbit/product-recommender-system 1

Implementation of two recommender systems for electronic products rated by users on Amazon

cbartondock/chaos 0

study of the global attractor for the standard map (kicked rotor)

yuanbit/bioconda-recipes 0

Conda recipes for the bioconda channel.

yuanbit/chat_app 0

Realtime single channel chat application with sentiment analysis implemented using Python Flask-SocketIO

push eventjina-ai/jina-hub

Joan Fontanals Martinez

commit sha 5ee32a77831089bf1db0430bf78eed96975b1596

feat: use single decorator for image crafters

view details

push time in 4 minutes

push eventjina-ai/jina

Joan Fontanals Martinez

commit sha 6cd5dbb2e62be5d540ae65c498aaf89863d14e1c

fix: extract required keys when decorator

view details

push time in 38 minutes

PR opened jina-ai/jina-hub

feat: use single decorator for image crafters
+128 -72

0 comment

21 changed files

pr created time in an hour

create barnchjina-ai/jina-hub

branch : image-crafters-single

created branch time in an hour

pull request commentjina-ai/jina

feat: adapt for crafter batching

Latency summary

Current PR yields:

  • 😶 index QPS at 979, delta to last 3 avg.: -3%
  • 😶 query QPS at 14, delta to last 3 avg.: -5%

Breakdown

Version Index QPS Query QPS
current 979 14
1.0.7 1015 14

Backed by latency-tracking. Further commits will update this comment.

JoanFM

comment created time in an hour

pull request commentjina-ai/jina

feat: adapt for crafter batching

Codecov Report

Merging #2126 (8ed2431) into master (f0b6a44) will decrease coverage by 61.91%. The diff coverage is 5.55%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master    #2126       +/-   ##
===========================================
- Coverage   89.90%   27.99%   -61.92%     
===========================================
  Files         211      189       -22     
  Lines       11209    10624      -585     
===========================================
- Hits        10078     2974     -7104     
- Misses       1131     7650     +6519     
Flag Coverage Δ
daemon ?
jina 27.99% <5.55%> (-62.39%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/drivers/__init__.py 65.85% <0.00%> (-28.02%) :arrow_down:
jina/drivers/craft.py 44.44% <0.00%> (-55.56%) :arrow_down:
jina/executors/crafters/__init__.py 60.00% <0.00%> (-40.00%) :arrow_down:
jina/types/sets/document.py 28.34% <33.33%> (-66.93%) :arrow_down:
jina/schemas/pod.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/parsers/base.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/parsers/flow.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/parsers/ping.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/schemas/flow.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/schemas/meta.py 0.00% <0.00%> (-100.00%) :arrow_down:
... and 187 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f0b6a44...e50c5c0. Read the comment docs.

JoanFM

comment created time in an hour

push eventjina-ai/jina-hub

Joan Fontanals Martinez

commit sha c4e81886bd1fc37314a3c86298409efbbe461041

feat: ensure single batch for audio crafters

view details

push time in an hour

push eventjina-ai/jina-hub

Joan Fontanals Martinez

commit sha 82fce78f042319c7cf342c019f8da87c120c1f7c

feat: ensure single batch for audio crafters

view details

push time in an hour

PR opened jina-ai/jina

feat: adapt for crafter batching

Changes introduced Ensure arguments are passed to executor in batches inside CraftDriver.

For now this is tested to work when craft is decorated with single.

TODO

  • [ ] Ensure every crafter in the hub is decorated with single and tested in batches.
  • [ ] Merge this PR
  • [ ] Slowly migrate crafters to use batching when deemed necessary.
+213 -41

0 comment

14 changed files

pr created time in an hour

push eventjina-ai/jina

Joan Fontanals Martinez

commit sha e50c5c03f285bf543b27d80faa237e20adaca2b2

fix: extract required keys when decorator

view details

push time in an hour

issue commentjina-ai/jina

incorrect usage of NotImplementedError

Yeah, so to address this issue we can search our codebase to make sure w use NotImplementedError correctly? Seems like a good-first-issue.

bwanglzu

comment created time in an hour

push eventjina-ai/jina

Joan Fontanals Martinez

commit sha 075780c7eeb6fa7a267a5d95843e9fe841011626

test: add integ tests for different crafters single and batching

view details

push time in an hour

pull request commentjina-ai/jina

test(encoders): add tests

Sorry, I forgot to request reviews again. @bwanglzu @JoanFM

Yongxuanzhang

comment created time in 2 hours

pull request commentjina-ai/jina

docs(jina): fix broken links in contributing doc instructions

Hi @atibaup, could you merge/rebase from master and push again?

atibaup

comment created time in 2 hours

push eventjina-ai/jina

YongxuanZhang

commit sha 69e92cc898a90aa48780358ec88c26decd9bca40

fix: black

view details

push time in 2 hours

push eventjina-ai/jina

Jina Dev Bot

commit sha 2f51048c3b2a1c7fb3cf4fe7c4d0fea19581fe5c

chore(style): reformatted by jina-dev-bot

view details

Jina Dev Bot

commit sha 4b6af96dec04ac17a9375f44c73ff72ac13c36ec

chore(contributor): update contributors

view details

cristian

commit sha cb40b44f05212dbf69f8ef40792d094e51553048

ci: include docstr linter (#2045)

view details

cristian

commit sha 3af29051e8e9e35eca5c76c52f86e60df515c834

feat(binarypb): delete on dump (#2102)

view details

Maximilian Werk

commit sha 9bbb0769b0474ddb5a0682b518f41f7e9643ff43

fix: expose env variable for workspace (#2114)

view details

CatStark

commit sha 7169fb56ad2fa3a919c0225e35fdb0ed08b910e4

fix: fix traversal_path, change from c to r (#2116)

view details

cristian

commit sha 640daf4d389768be216dac4125ef4e837ee65d23

ci: add black (#2036) * ci: add black * ci: add git blame

view details

Jina Dev Bot

commit sha e01e57df00deda8ea7bbda1f0a26ba25c60782a6

chore(contributor): update contributors

view details

cristian

commit sha dc2be2f009b8e82be8241363efd71ce3f32cbf84

ci: reenable docstrings lint (#2118)

view details

cristian

commit sha c258e4aa22495d3809ecbcb0ee9966938ccdbfe5

docs: update black docs and sha (#2117)

view details

Jina Dev Bot

commit sha 7dd876d0a1fbfca3818c13a68521b80e43a1c617

chore(contributor): update contributors

view details

Jina Dev Bot

commit sha 47ac7b0a8d55faf8032579cb6e114e9b02bf392f

chore(version): the next version will be 1.0.9 build(hanxiao): Sunday night weekly patch release

view details

Jina Dev Bot

commit sha 3e91a1faf4680e055aff4cf1d6db637024aa5612

chore(contributor): update contributors

view details

Jina Dev Bot

commit sha 5e32eddc940b3259fe712d3fffeecc96c8e23afb

Merge remote-tracking branch 'origin/master'

view details

Jina Dev Bot

commit sha 666d302ef490d35f7eb080f108994e4582c59dc2

chore(docs): update TOC

view details

Han Xiao

commit sha dd687735bb2c569f8dee51ff262d88b3f271b681

refactor(cli): rename silent to quiet (#2122)

view details

Deepankar Mahapatro

commit sha b429d2215475e56a8808b5687db9e90c2d1e133e

feat(schema): generate pydantic based jsonschema for any jina proto (#2121) * feat(schema): genereate pydantic based jsonschema for any jina proto * docs: fix return type * docs: fix docstrings * feat(schema): camel case support for all fields * test(schema): jina document to pydantic document * fix(schema): remove proto name check

view details

Florian Hönicke

commit sha caae3f6d9ba9e29583f08a7d721f8a1629e171fa

refactor: crud delete types (#2014)

view details

Joan Fontanals

commit sha f0b6a44045f7fccca05a34020fd42981ce34dc4e

refactor: prepare changes to have batching for every executor (#2110) Co-authored-by: Nan Wang <nan.wang@jina.ai>

view details

YongxuanZhang

commit sha b57bb1814a85fade152219979bb369b8cfdc2d3c

Merge branch 'master' into test-encoders-framework

view details

push time in 3 hours

push eventjina-ai/jina

Joan Fontanals Martinez

commit sha 09fffbd786227d084d1d205fab75cdbf19ef2606

test: add single to tests

view details

push time in 3 hours

Pull request review commentjina-ai/jina

Improve Formatter

 def format(self, record):         :param record: A LogRecord object         :returns: Formatted LogRecord with level-colour MAPPING to add corresponding colour.         """-        cr = copy(record)+        cr = deepcopy(record)

Hi @chunyuema, will deepcopy harm the performance?

chunyuema

comment created time in 3 hours

PR opened jina-ai/jina-hub

feat: increase verbosity for zarr indexer
  • Add usage for ZarrIndexer
+58 -3

0 comment

4 changed files

pr created time in 3 hours

create barnchjina-ai/jina-hub

branch : zarr-doc

created branch time in 3 hours

Pull request review commentjina-ai/jina-hub

feat: CLIP zero shot classifier

+from typing import List++import torch+from jina.executors.classifiers import BaseClassifier+from jina.executors.decorators import as_ndarray, batching+from jina.executors.devices import TorchDevice+from PIL import Image+++class CLIPZeroShotClassifier(TorchDevice, BaseClassifier):+    """+    :class:`ClipZeroShotClassifier` Zero Shot classification for images using OpenAI Clip.++    Internally, :class:`ClipZeroShotClassifier` wraps the `CLIP` modeL from https://github.com/openai/CLIP+    :param labels: labels for the classification task. +    :param model_name: The name of the model. Supported models include ``ViT-B/32`` and ``RN50``.+    :param args: Additional positional arguments.+    :param kwargs: Additional positional arguments.+    """++    def __init__(self, labels: List[str], model_name: str ='ViT-B/32',

good finding, we need to add this definitely

deepampatel

comment created time in 3 hours

PR opened jina-ai/jina-hub

feat: increase verbosity for SpTag Indexer
  • Add usage for SpTagIndexer.
+82 -3

0 comment

4 changed files

pr created time in 3 hours

create barnchjina-ai/jina-hub

branch : sptag-doc

created branch time in 3 hours

Pull request review commentjina-ai/jina-hub

feat: CLIP zero shot classifier

+from typing import List++import torch+from jina.executors.classifiers import BaseClassifier+from jina.executors.decorators import as_ndarray, batching+from jina.executors.devices import TorchDevice+from PIL import Image+++class CLIPZeroShotClassifier(TorchDevice, BaseClassifier):+    """+    :class:`ClipZeroShotClassifier` Zero Shot classification for images using OpenAI Clip.++    Internally, :class:`ClipZeroShotClassifier` wraps the `CLIP` modeL from https://github.com/openai/CLIP+    :param labels: labels for the classification task. +    :param model_name: The name of the model. Supported models include ``ViT-B/32`` and ``RN50``.+    :param args: Additional positional arguments.+    :param kwargs: Additional positional arguments.+    """++    def __init__(self, labels: List[str], model_name: str ='ViT-B/32',

@JoanFM i will add default labels. Seems the ci hub build is not called because classifiers folder is not tracked https://github.com/jina-ai/jina-hub/blob/master/.github/workflows/ci.yml#L29

deepampatel

comment created time in 3 hours

Pull request review commentjina-ai/jina

refactor: refactor rankers, move logic to driver

 def _apply_all(self, docs: 'DocumentSet', *args, **kwargs) -> None:             )              matches = doc.matches-            old_match_scores = {match.id: match.score.value for match in matches}-            match_meta = (-                {match.id: match.get_attrs(*self._exec_match_keys) for match in matches}-                if self._exec_match_keys-                else None-            )+            num_matches = len(matches)+            old_match_scores = []+            needs_match_meta = self._exec_match_keys is not None+            match_meta = [] if needs_match_meta else None+            for match in matches:+                old_match_scores.append(match.score.value)+                if needs_match_meta:+                    match_meta.append(match.get_attrs(*self._exec_match_keys))              # if there are no matches, no need to sort them             if not old_match_scores:                 continue -            new_match_scores = self.exec_fn(query_meta, old_match_scores, match_meta)-            self._sort_matches_in_place(doc, new_match_scores)+            new_scores = self.exec_fn(old_match_scores, query_meta, match_meta)+            if num_matches != len(new_scores):+                msg = (+                    f'The number of matches to be scored {num_matches} do not match the number of scores returned '+                    f'by the ranker {self.exec.__name__} '+                )+                self.logger.error(msg)+                raise ValueError(msg)+            self._sort_matches_in_place(matches, new_scores)      def _sort_matches_in_place(-        self, context_doc: 'Document', match_scores: 'np.ndarray'+        self, matches: 'MatchSet', match_scores: Iterable[float]     ) -> None:         op_name = self.exec.__class__.__name__-        cm = context_doc.matches-        cm.build()-        for match_id, score in match_scores:-            cm[match_id].score = NamedScore(-                value=score, op_name=op_name, ref_id=context_doc.id-            )+        ref_doc_id = matches._ref_doc.id++        for match, score in zip(matches, match_scores):+            match.score = NamedScore(value=score, op_name=op_name, ref_id=ref_doc_id) -        cm.sort(key=lambda x: x.score.value, reverse=True)+        matches.sort(key=lambda x: x.score.value, reverse=True)

We do not have a reverse parameter. I think for now every Ranker is scoring per relevance so it is fair to set it to True

JoanFM

comment created time in 4 hours

Pull request review commentjina-ai/jina

refactor: refactor rankers, move logic to driver

 def __init__(         reverse: bool = False,         traversal_paths: Tuple[str] = ('r',),         *args,-        **kwargs+        **kwargs,

I guess is just black formatting?

JoanFM

comment created time in 4 hours

push eventjina-ai/jina

Joan Fontanals

commit sha c93f59a653f963cba445ba87aa6ec10c6d0b5c33

fix: update jina/drivers/rank/aggregate/__init__.py Co-authored-by: Nan Wang <nan.wang@jina.ai>

view details

push time in 4 hours

pull request commentjina-ai/jina

docs(jina): fix broken links in contributing doc instructions

Latency summary

Current PR yields:

  • 🐢🐢 index QPS at 1024, delta to last 3 avg.: -7%
  • 😶 query QPS at 18, delta to last 3 avg.: -5%

Breakdown

Version Index QPS Query QPS
current 1024 18
1.0.8 1128 19
1.0.7 1091 18

Backed by latency-tracking. Further commits will update this comment.

atibaup

comment created time in 5 hours

pull request commentjina-ai/jina

docs(jina): fix broken links in contributing doc instructions

Codecov Report

Merging #2125 (3912093) into master (f0b6a44) will decrease coverage by 59.91%. The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           master    #2125       +/-   ##
===========================================
- Coverage   89.90%   29.99%   -59.92%     
===========================================
  Files         211      208        -3     
  Lines       11209    11177       -32     
===========================================
- Hits        10078     3353     -6725     
- Misses       1131     7824     +6693     
Flag Coverage Δ
daemon 77.89% <ø> (+27.43%) :arrow_up:
jina 27.46% <ø> (-62.92%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
jina/schemas/pod.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/parsers/base.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/parsers/flow.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/parsers/ping.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/schemas/flow.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/schemas/meta.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/docker/helper.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/parsers/client.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/schemas/driver.py 0.00% <0.00%> (-100.00%) :arrow_down:
jina/parsers/hub/new.py 0.00% <0.00%> (-100.00%) :arrow_down:
... and 167 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f0b6a44...3912093. Read the comment docs.

atibaup

comment created time in 5 hours

pull request commentjina-ai/jina

docs(jina): fix broken link in contributing doc instructions

Latency summary

Current PR yields:

  • 😶 index QPS at 917, delta to last 3 avg.: -3%
  • 😶 query QPS at 14, delta to last 3 avg.: -1%

Breakdown

Version Index QPS Query QPS
current 917 14
1.0.8 940 14
1.0.7 953 14

Backed by latency-tracking. Further commits will update this comment.

atibaup

comment created time in 5 hours