The path to the fetch_faster_rcnn_models.sh script in the demo.py helper message is incorrect. This PR fixes the typo so that users who run the demo before downloading models will be able to copy the path to the script and run it.

I may have misunderstood the question, but inputs are provided by users, therefore users already have information about input tensor and their dimensions.

How can I access the tensors (such as pred_mask) contained in the object Instances (the output of the predictor)? I need to use the tensors to further calculations and I don't know how I can extract them.

You can find these info in documentation: https://detectron2.readthedocs.io/modules/structures.html#detectron2.structures.Instances

❓ Questions and Help

Thanks for offering this api. I can't open the website https://www.lvisdataset.org/,so I want to ask you how to get the dataset by this api or the instruction of this api. Thank you !!!

The correct website link is: https://www.lvisdataset.org You can't download the dataset via the API. You can download the dataset here: https://www.lvisdataset.org/dataset Example usage for evaluation can be found here: https://github.com/lvis-dataset/lvis-api/blob/master/test.py

❓ How to do something using detectron2

Describe what you want to do, including:

1. what inputs you will provide, if any:
2. what outputs you are expecting:

❓ What does an API do and how to use it?

Please link to which API or documentation you're asking about from https://detectron2.readthedocs.io/

For meaning of a config, please see https://detectron2.readthedocs.io/modules/config.html#config-references

Code:

The following code gives the output and I want the input tensor and input dims as well. Is there any changes required

from caffe2.python import workspace import numpy as np from detectron2.export import Caffe2Model import torch

print(torch.version)

import cv2 img = cv2.imread('test.png') image = torch.as_tensor(img.astype("float32").transpose(2, 0, 1)) data = {'image': image, 'height': image.shape[1], 'width': image.shape[2]} caffe2_model = Caffe2Model.load_protobuf('.') output = caffe2_model([data]) print('\nOuput:\n',output)

NOTE:

print(torch.version) print ("Creating random data") data = np.random.rand(3, 2) #print(data) print ("Adding data to workspace ...") workspace.FeedBlob("mydata", data) print ("Retrieving data from workspace") mydata = workspace.FetchBlob("mydata") #print(mydata)

import cv2 img = cv2.imread('test.png') image = torch.as_tensor(img.astype("float32").transpose(2, 0, 1)) data = {'image': image, 'height': image.shape[1], 'width': image.shape[2]} caffe2_model = Caffe2Model.load_protobuf('.') output = caffe2_model([data]) print('\nOuput:\n',output)

How can I access the tensors (such as pred_mask) contained in the object Instances (the output of the predictor)? I need to use the tensors to further calculations and I don't know how I can extract them.

Same issue, the bucket might have been removed

There's a few differences between this repo and the published package including a common folder that includes gason.cpp gason.h maskApi.c maskApi.h and these changes:

--- pycocotools-2.0.2/pycocotools/coco.py       2020-08-14 19:41:27.000000000 +0100
+++ cocoapi/PythonAPI/pycocotools/coco.py       2020-12-08 15:44:57.804798462 +0000
@@ -81,8 +81,7 @@
         if not annotation_file == None:
             print('loading annotations into memory...')
             tic = time.time()
-            with open(annotation_file, 'r') as f:
-                dataset = json.load(f)
+            dataset = json.load(open(annotation_file, 'r'))
             assert type(dataset)==dict, 'annotation file format {} not supported'.format(type(dataset))
             print('Done (t={:0.2f}s)'.format(time.time()- tic))
             self.dataset = dataset
@@ -315,8 +314,7 @@
         print('Loading and preparing results...')
         tic = time.time()
         if type(resFile) == str or (PYTHON_VERSION == 2 and type(resFile) == unicode):
-            with open(resFile) as f:
-                anns = json.load(f)
+            anns = json.load(open(resFile))
         elif type(resFile) == np.ndarray:
             anns = self.loadNumpyAnnotations(resFile)
         else:
@@ -440,4 +438,4 @@
         """
         rle = self.annToRLE(ann)
         m = maskUtils.decode(rle)
-        return m
+        return m
\ No newline at end of file

Currently if you run pip install pycocotools on Python 3.6 (or earlier) you will see something like this:

ERROR: Command errored out with exit status 1:
[...]
      File "/tmp/easy_install-i3fx38y0/numpy-1.20.0rc1/setup.py", line 30, in <module>
        package_dir={'pycocotools': 'pycocotools'},
    RuntimeError: Python version >= 3.7 required.

[email protected] came out a few days ago but I couldn't quite figure out why it was being installed.

I unpacked the release of pycocotools from pypi and found this in the setup.py

setup_requires = [
    'setuptools>=18.0',
    'cython>=0.27.3',
    'numpy',
]
dist.Distribution().fetch_build_eggs(setup_requires)

import numpy as np

ext_modules = [
    Extension(
        'pycocotools._mask',
        sources=['./common/maskApi.c', 'pycocotools/_mask.pyx'],
        include_dirs=[np.get_include(), './common'],
        extra_compile_args=[] if platform.system() == 'Windows' else ['-Wno-cpp', '-Wno-unused-function', '-std=c99'],
    )
]

I could see that there is an open dependency on numpy, so I tried to look at the source and found different contents which were last changed 3 years ago: https://github.com/cocodataset/cocoapi/blob/8c9bcc3cf640524c4c20a9c40e89cb6a2f2fa0e9/PythonAPI/setup.py#L1-L27

Possibly related to #416 -- what is going on with the published package on pypi?

+1

A great explanation of the coco file format along with detailed explanation of RLE and iscrowd - Coco file format

❓ Questions and Help

Thanks for offering this api. I can't open the website https://www.lvisdataset.org/,so I want to ask you how to get the dataset by this api or the instruction of this api. Thank you !!!

❓ How to do something using detectron2

Describe what you want to do, including:

1. what inputs you will provide, if any:

• A detectron2 Mask R-CNN model

2. what outputs you are expecting:

• An exported Caffe format model
• It would be great to have a reference for how to do this and how to use the generated model

❓ What does an API do and how to use it?

Please link to which API or documentation you're asking about from https://detectron2.readthedocs.io/

• https://detectron2.readthedocs.io/tutorials/deployment.html

For meaning of a config, please see https://detectron2.readthedocs.io/modules/config.html#config-references

NOTE:

1. Only general answers are provided. If you want to ask about "why X did not work" for something you did, please use the Unexpected behaviors issue template.

2. About how to implement new models / new dataloader / new training logic, etc., check documentation first.

3. We do not answer machine learning / computer vision questions that are not specific to detectron2, such as how a model works, how to improve your training/make it converge, or what algorithm/methods can be used to achieve X.

We will not support converting a Mask R-CNN to caffe format.

Hello，Pointrend：why not update maskloss and only update maskpointloss？thanks！

This is not among the types of issues we answer.

❓ How to do something using detectron2

Describe what you want to do, including:

1. what inputs you will provide, if any:

• A detectron2 Mask R-CNN model

2. what outputs you are expecting:

• An exported Caffe format model
• It would be great to have a reference for how to do this and how to use the generated model

❓ What does an API do and how to use it?

Please link to which API or documentation you're asking about from https://detectron2.readthedocs.io/

• https://detectron2.readthedocs.io/tutorials/deployment.html

For meaning of a config, please see https://detectron2.readthedocs.io/modules/config.html#config-references

NOTE:

1. Only general answers are provided. If you want to ask about "why X did not work" for something you did, please use the Unexpected behaviors issue template.

2. About how to implement new models / new dataloader / new training logic, etc., check documentation first.

3. We do not answer machine learning / computer vision questions that are not specific to detectron2, such as how a model works, how to improve your training/make it converge, or what algorithm/methods can be used to achieve X.

If you need help to solve an unexpected issue you observed, please include details following the "Unexpected behaviors" issue template.

Sorry, I've got it. But I have a new question. When I added the pointrend module to blendmask, I found that the mask loss of blendmask was falling more slowly and get worse segmentation accuracy. 1.blendmask to train:89.2% 2.blendmask and pointrend to train, only blendmask to predict:89.0% 3.blendmask and pointrend to train, blendmask and pointrend to predict:88.5% Do you know why? I would appreciate it if you could let me know. Thanks！

Good afternoon, my question is how can I get the outline of each object. The documentation says: “pred_masks”: a Tensor of shape (N, H, W), masks for each detected instance. In practice, I only get an array of False.

But if I output the self-mask from the Visualizer function, I get a normal data array from which I can take the contour I need.

Should it be so? And how do I get the outline of each object? I did not find an answer in the documentation.

cfg = get_cfg()
cfg.MODEL.DEVICE=("cpu")
cfg.merge_from_file("../configs/COCO-PanopticSegmentation/panoptic_fpn_R_101_3x.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
cfg.MODEL.WEIGHTS = "detectron2://COCO-PanopticSegmentation/panoptic_fpn_R_101_3x/139514519/model_final_cafdb1.pkl"

predictor = DefaultPredictor(cfg)
im = cv2.imread("1.png")
outputs = predictor(im)
class_names = MetadataCatalog.get(cfg.DATASETS.TRAIN[0]).stuff_classes
print(np.array(outputs["instances"].pred_masks)) #return array where only 'False'

The array after executing my code looks like this: [False False False ... False False False] [False False False ... False False False] [False False False ... False False False] [False False False ... False False False]...

Thank you very much for your help. I wrote a new script. Now I can display the mask of each found object in a separate file. Everything worked out!

   outputs = predictor(im)
    s=np.array(outputs["sem_seg"])
    s=(s.astype(int))
    print(s)
    s=(np.where(s>=1, 255, s))
    acount=0
    for yi in s:
        cv2.imwrite(str(acount)+'maske.jpg', yi)
        acount+=1