profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/slashdd/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Eric Desrochers slashdd Canonical Ltd Sherbrooke,QC, Canada

soscleaner/soscleaner 41

A python application to clean sensitive and un-wanted data from an existing sosreport

slashdd/charm-keystone 0

Juju Charm - Keystone. Mirror of code maintained at opendev.org.

slashdd/hotsos 0

Extract application-specific information from a sosreport or host.

slashdd/landscape-client 0

The Landscape Client is the agent which communicates with the Landscape service.

slashdd/maas-docs 0

Documentation for MAAS

slashdd/netplan 0

Backend-agnostic network configuration in YAML

slashdd/nova 0

OpenStack Compute (Nova)

slashdd/pcp 0

Performance Co-Pilot

slashdd/sos 0

A unified tool for collecting system logs and other debug information

push eventslashdd/sos

Eric Desrochers

commit sha 921218bdefad1ad95bea2dbc639459f26a5f6113

[report] check for symlink before rmtree when opt estimate-only is use Check if the dir is also symlink before performing rmtree() method so that unlink() method can be used instead. Traceback (most recent call last): File "./bin/sos", line 22, in <module> sos.execute() File "/tmp/sos/sos/__init__.py", line 186, in execute self._component.execute() OSError: Cannot call rmtree on a symbolic link Closes: #2727 Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>

view details

push time in 3 days

push eventslashdd/sos

Eric Desrochers

commit sha abfe158c198e8a80e29e15199085ca4ebf114299

[report] check for symlink before rmtree when opt estimate-only is use Check if the dir is also symlink before performing using rmtree() method, it not, then unlink() should be used. Call Trace: Finished running plugins Traceback (most recent call last): File "./bin/sos", line 22, in <module> sos.execute() File "/tmp/sos/sos/__init__.py", line 186, in execute self._component.execute() OSError: Cannot call rmtree on a symbolic link > /usr/lib/python3.8/shutil.py(726)rmtree() -> raise OSError("Cannot call rmtree on a symbolic link") Closes: #2727 Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>

view details

push time in 3 days

pull request commentsosreport/sos

[report] check for symlink before rmtree when opt estimate-only is use

With the PR, no more complaints about rmtree() method on symlink.

sudo python3 ./bin/sos report -o systemd --estimate-only

sosreport (version 4.2)

WARNING: unable to set option for disabled or non-existing plugin (networking).

This command will collect system configuration and diagnostic
information from this Ubuntu system.

For more information on Canonical visit:

        Community Website  : https://www.ubuntu.com/
        Commercial Support : https://www.canonical.com

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.



Estimate-only mode enabled, which overrides some options:
  --threads=4 overriden to 1
  --build enabled
  --no-postproc enabled

Press ENTER to continue, or CTRL-C to quit.

Optionally, please enter the case id that you are generating this report for []: 

 Setting up archive ...
 Setting up plugins ...
 Running plugins. Please wait ...

  Starting 1/1   systemd         [Running: systemd]                                       
  Finished running plugins                                                               
Skipping postprocessing of collected data

Your sosreport build tree has been generated in:
	/tmp/sosreport-nuc7-2021-10-19-cwcosbe


Estimated disk space requirement for whole uncompressed sos report directory: 4.41MiB
Three biggest plugins:  systemd: 4MiB,  sos_logs_reports: 540KiB

Please note the estimation is relevant to the current options.
Be aware that the real disk space requirements might be different.
slashdd

comment created time in 3 days

PR opened sosreport/sos

[report] check for symlink before rmtree when opt estimate-only is use

Check if the dir is also symlink before performing using rmtree() method, it not, then unlink() should be used.

Call Trace: Finished running plugins Traceback (most recent call last): File "./bin/sos", line 22, in <module> sos.execute() File "/tmp/sos/sos/init.py", line 186, in execute self._component.execute() OSError: Cannot call rmtree on a symbolic link

/usr/lib/python3.8/shutil.py(726)rmtree() -> raise OSError("Cannot call rmtree on a symbolic link")

Resolves: #2727

Signed-off-by: Eric Desrochers eric.desrochers@canonical.com


Please place an 'X' inside each '[]' to confirm you adhere to our Contributor Guidelines

  • [ ] Is the commit message split over multiple lines and hard-wrapped at 72 characters?
  • [ ] Is the subject and message clear and concise?
  • [ ] Does the subject start with [plugin_name] if submitting a plugin patch or a [section_name] if part of the core sosreport code?
  • [ ] Does the commit contain a Signed-off-by: First Lastname email@example.com?
  • [ ] Are any related Issues or existing PRs properly referenced via a Closes (Issue) or Resolved (PR) line?
+1 -1

0 comment

1 changed file

pr created time in 3 days

push eventslashdd/sos

Eric Desrochers

commit sha 300e43bc5b47f8d479f2711968d335423d1e599b

[report] check for symlink before rmtree when opt estimate-only is use Check if the dir is also symlink before performing using rmtree() method, it not, then unlink() should be used. Call Trace: Finished running plugins Traceback (most recent call last): File "./bin/sos", line 22, in <module> sos.execute() File "/tmp/sos/sos/__init__.py", line 186, in execute self._component.execute() OSError: Cannot call rmtree on a symbolic link > /usr/lib/python3.8/shutil.py(726)rmtree() -> raise OSError("Cannot call rmtree on a symbolic link") Resolves: #2727 Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>

view details

push time in 3 days

create barnchslashdd/sos

branch : estimate

created branch time in 3 days

fork slashdd/sos

A unified tool for collecting system logs and other debug information

http://sos.rtfd.org

fork in 3 days

issue commentsosreport/sos

[report] estimate struggles with plugins that capture symlinks

           for f in os.listdir(self.archive.get_tmp_dir()):
                f = os.path.join(self.archive.get_tmp_dir(), f)
-                 if os.path.isdir(f):
+                if os.path.isdir(f) and not os.path.islink(f):
                    rmtree(f)
                else:
                    os.unlink(f)
        return True

slashdd

comment created time in 3 days

issue commentsosreport/sos

[report] estimate struggles with plugins that capture symlinks

report/init.py

1060             for f in os.listdir(self.archive.get_tmp_dir()):
1061                 f = os.path.join(self.archive.get_tmp_dir(), f)
1062                 if os.path.isdir(f):
1063                     rmtree(f)
1064                 else:
1065                     os.unlink(f)
1066         return True

I have created a quick test (outisde sosreport) as follows:

lrwxrwxrwx  1 root   root        4 Oct 19 15:58 test1 -> test
drwxr-xr-x  2 root   root     4096 Oct 19 15:58 test

# python3
Python 3.8.10 (default, Sep 28 2021, 16:10:42) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shutil
>>> import os
>>> os.path.isdir('/tmp/test1')
True
>>> shutil.rmtree('/tmp/test1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/shutil.py", line 728, in rmtree
    onerror(os.path.islink, path, sys.exc_info())
  File "/usr/lib/python3.8/shutil.py", line 726, in rmtree
    raise OSError("Cannot call rmtree on a symbolic link")
OSError: Cannot call rmtree on a symbolic link

It seems like the validation made by isdir() isn't appropriate to catch when to unlink() symlink. We may want to add extra check such as islink() or else.

slashdd

comment created time in 3 days

issue openedsosreport/sos

[report] estimate struggles with plugins that capture symlinks

--estimate-only doesn't work on plugins containing symbolic link

Plugin without symlink (find . -type l -ls)

# sudo python3 ./bin/sos report --debug -o ubuntu --estimate-only

sosreport (version 4.2)

WARNING: unable to set option for disabled or non-existing plugin (networking).

This command will collect system configuration and diagnostic
information from this Ubuntu system.

For more information on Canonical visit:

        Community Website  : https://www.ubuntu.com/
        Commercial Support : https://www.canonical.com

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.



Estimate-only mode enabled, which overrides some options:
  --threads=4 overriden to 1
  --build enabled
  --no-postproc enabled

Press ENTER to continue, or CTRL-C to quit.

Optionally, please enter the case id that you are generating this report for []: 

 Setting up archive ...
 Setting up plugins ...
 Running plugins. Please wait ...

  Starting 1/1   ubuntu          [Running: ubuntu]                                        
  Finished running plugins                                                               
Skipping postprocessing of collected data

Your sosreport build tree has been generated in:
	/tmp/sosreport-nuc7-2021-10-19-oqpbufe


Estimated disk space requirement for whole uncompressed sos report directory: 7.10KiB
Three biggest plugins:  sos_logs_reports: 5KiB,  ubuntu: 2KiB

Please note the estimation is relevant to the current options.
Be aware that the real disk space requirements might be different.

Plugin with symlink (find . -type l -ls)

# sudo python3 ./bin/sos report --debug -o systemd --estimate-only

sosreport (version 4.2)

WARNING: unable to set option for disabled or non-existing plugin (networking).

This command will collect system configuration and diagnostic
information from this Ubuntu system.

For more information on Canonical visit:

        Community Website  : https://www.ubuntu.com/
        Commercial Support : https://www.canonical.com

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.



Estimate-only mode enabled, which overrides some options:
  --threads=4 overriden to 1
  --build enabled
  --no-postproc enabled

Press ENTER to continue, or CTRL-C to quit.

Optionally, please enter the case id that you are generating this report for []: 

 Setting up archive ...
 Setting up plugins ...
 Running plugins. Please wait ...

  Starting 1/1   systemd         [Running: systemd]                                       

  Finished running plugins                                                               Traceback (most recent call last):
  File "./bin/sos", line 22, in <module>
    sos.execute()
  File "/tmp/sos/sos/__init__.py", line 186, in execute
    self._component.execute()
OSError: Cannot call rmtree on a symbolic link

> /usr/lib/python3.8/shutil.py(726)rmtree()
-> raise OSError("Cannot call rmtree on a symbolic link")
(Pdb) 

created time in 3 days

pull request commentcanonical/hotsos

Alert if kernel is marked as tainted

Is there perhaps another way to determine if the kernel is tainted other than looking in kern.log? e.g would uname -a show that too? If so i'd prefer to add that to core.plugins.kernel as a field in the summary e.g.:

Yes, we can query the tainted state by reading /proc/sys/kernel/tainted instead. If that returns 0, the kernel is not tainted.

And I just checked, sosreport captures it as part of the kernel plugin: https://github.com/sosreport/sos/blob/main/sos/report/plugins/kernel.py#L105

kernel: version: 5.0.0-37-generic tainted: true boot: ro quiet splash vt.handoff=1 ...

I'm +1 on the Tainted: true proposal, instead of a KernelWarning.

slashdd

comment created time in 7 days

push eventslashdd/hotsos

Eric Desrochers

commit sha a503e6e417cff3e1bd0db38324643586ab776475

Alert if kernel is marked as tainted Indicates if running kernel mark itself as 'tainted' due to out-of-tree, proprietary, ... modules. If kernel is tainted: ... CPU: XX PID: YYY Comm: migration/27 Tainted: G OX ... If not tainted: CPU: X PID: YYY Comm: khungtaskd Not tainted Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>

view details

push time in 8 days

push eventslashdd/hotsos

Eric Desrochers

commit sha 1db6eeb090795d83af07f206149168f93d98a2d8

Alert if kernel is marked as tainted Indicates if running kernel mark itself as 'tainted' due to out-of-tree, proprietary, ... modules. If kernel is tainted: ... CPU: XX PID: YYY Comm: migration/27 Tainted: G OX ... If not tainted: CPU: X PID: YYY Comm: khungtaskd Not tainted Signed-off-by: Eric Desrochers <eric.desrochers@canonical.com>

view details

push time in 8 days

PR opened canonical/hotsos

Alert if kernel is marked as tainted

Indicates if running kernel mark itself as 'tainted' due to out-of-tree, proprietary, ... modules.

If kernel is tainted: ... CPU: XX PID: YYY Comm: migration/27 Tainted: G OX ...

If not tainted: CPU: X PID: YYY Comm: khungtaskd Not tainted

Signed-off-by: Eric Desrochers eric.desrochers@canonical.com

+12 -0

0 comment

3 changed files

pr created time in 8 days

create barnchslashdd/hotsos

branch : kerneltaint

created branch time in 8 days

fork slashdd/hotsos

Extract application-specific information from a sosreport or host.

fork in 8 days

issue commentsosreport/sos

[4.0] Add Snap and Flatpak support

Main discussion: https://github.com/sosreport/sos/issues/2237

TurboTurtle

comment created time in 17 days

IssuesEvent

issue commentsosreport/sos

[4.0] Add Snap and Flatpak support

This issue is not fixed. The intent of this bug was to make sure sos support SNAP FlatPak in PackageManager() just like it does for RPM and DEB.

The PR suggested by @arif-ali is the configuration to build sos as a SNAP, which is a different thing.

TurboTurtle

comment created time in 17 days

issue commentsosreport/sos

PackageManager support, how we want it to evolve

I'm just gonna poke this issue with a stick, I have a situation like this:

System: Ubuntu 20.04 running on Google Cloud Platform

My GCP plugin I'm working on, wants to collect gcloud output. gcloud is traditionally provided in a package called google-cloud-sdk - however, on this instance this package in installed via snap and not apt/dpkg.

While we are still working on the PackageManager to support FlatPak and SNAP, you could add (PR) the following commands = ('gcloud',) inside your plugin. Just make sure the binary is found in the Ubuntu policy defined PATH env: https://github.com/sosreport/sos/blob/main/sos/policies/distros/ubuntu.py#L22-L23 .

As you can see, currently with Ubuntu and their snaps, there is no one right way of installation. Can I use SoSPredicate(self, packages=['google-cloud-sdk']) in a way that would check for snap installs too?

BryanQuigley

comment created time in 17 days

Pull request review commentsosreport/sos

[ceph] split the ceph plugin into multiple plugins

+# This file is part of the sos project: https://github.com/sosreport/sos+#+# This copyrighted material is made available to anyone wishing to use,+# modify, copy, or redistribute it subject to the terms and conditions of+# version 2 of the GNU General Public License.+#+# See the LICENSE file in the source distribution for further information.++from sos.report.plugins import Plugin, RedHatPlugin, UbuntuPlugin+from socket import gethostname+import re,os+++class CephRGW(Plugin, RedHatPlugin, UbuntuPlugin):++    short_desc = 'CEPH rgw'++    plugin_name = 'ceph_rgw'+    profiles = ('storage', 'virt', 'container')

Please add webserver profile in order to enable apache2 plugin (Debian/Ubuntu: apache2, RedHat: httpd)

https://docs.ceph.com/en/latest/man/8/radosgw/

nkshirsagar

comment created time in a month

PullRequestReviewEvent
PullRequestReviewEvent

issue commentsosreport/sos

[debian] debian policy knows only about wheezy and jessie.

We could definitely do a plugin review (and see if other plugins could benefit it). If it turned out to be only use/"needed" by the python plugin in the RedHat context, and it can easily be sorted it out without it. I'm in favor to drop it with the rationale that anyway most plugins are mostly backward compatible with no longer existing feature/cmd/tools/...

slashdd

comment created time in a month

issue commentsosreport/sos

[debian] debian policy knows only about wheezy and jessie.

Is this for Policy.dist_version()? Yes Right now only one place in the entire project uses this - the python plugin (and specifically only the RedHatPython plugin at that).

Should we look at better utilizing distro versioning or perhaps removing it? Maybe removing it would work I guess, if unused, depending on much Red Hat depends on it.

slashdd

comment created time in a month

issue openedsosreport/sos

[debian] debian policy knows only about wheezy and jessie.

The next release of Debian is codenamed bookworm — testing — no release date has been set
Debian 11 (bullseye) — current stable release
Debian 10 (buster) — current oldstable release
Debian 9 (stretch) — oldoldstable release, under LTS support
Debian 8 (jessie) — archived release, under extended LTS support
Debian 7 (wheezy) — obsolete stable release
Debian 6.0 (squeeze) — obsolete stable release

created time in a month

PullRequestReviewEvent

issue closedsosreport/sos

[4.2] avocado testsuite on Ubuntu

Either with avocado-framework (90.0) (debianized) or simple pip3 install avocado-framework (91.0), I get the following error for all tests un Ubuntu:

Same result using PPA or local fakeroot (w/ python 3.8 and 3.9) Reproducible everytime (tested in Focal/20.04LTS and Impish (Active devel release)

======================================================================
ERROR: 0-CleanHelpTest
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/sos/sos-4.2/.pybuild/cpython3_3.9_sosreport/build/tests/sos_tests.py", line 244, in setUp
    self._execute_sos_cmd()
  File "/var/tmp/sos/sos-4.2/.pybuild/cpython3_3.9_sosreport/build/tests/sos_tests.py", line 123, in _execute_sos_cmd
    self.cmd_output = process.run(exec_cmd, timeout=self.sos_timeout)
  File "/usr/local/lib/python3.9/dist-packages/avocado/utils/process.py", line 1079, in run
    cmd_result = sp.run(timeout=timeout)
  File "/usr/local/lib/python3.9/dist-packages/avocado/utils/process.py", line 952, in run
    self._init_subprocess()
  File "/usr/local/lib/python3.9/dist-packages/avocado/utils/process.py", line 666, in _init_subprocess
    raise details
  File "/usr/local/lib/python3.9/dist-packages/avocado/utils/process.py", line 659, in _init_subprocess
    self._popen = subprocess.Popen(cmd,
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory (/var/tmp/sos/sos-4.2/.pybuild/cpython3_3.9_sosreport/build/bin/sos clean --help): '/var/tmp/sos/sos-4.2/.pybuild/cpython3_3.9_sosreport/build/bin
...
======================================================================
ERROR: 0-StageOneReportTest
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/sos/sos-4.2/.pybuild/cpython3_3.8_sosreport/build/tests/sos_tests.py", line 410, in setUp
    super(BaseSoSReportTest, self).setUp()
  File "/var/tmp/sos/sos-4.2/.pybuild/cpython3_3.8_sosreport/build/tests/sos_tests.py", line 247, in setUp
    with open(os.path.join(self.tmpdir, 'output'), 'rb') as pfile:
FileNotFoundError: [Errno 2] No such file or directory: '/var/tmp/sos_tests.py.StageOneReportTest/output'

closed time in a month

slashdd

issue commentsosreport/sos

[4.2] avocado testsuite on Ubuntu

I had to reformat as follows for debian/rules:

 override_dh_auto_test:
	avocado run --test-runner=runner -t stageone tests/cleaner_tests
 	avocado run --test-runner=runner -t stageone tests/collect_tests
 	avocado run --test-runner=runner -t stageone tests/report_tests
 	avocado run --test-runner=runner -t stageone tests/vendor_tests

It works if I run the build locally using fakeroot. Falling when sending to a PPA.

I debug what could differ in a PPA. I think I have enough to continue.

Thanks !

slashdd

comment created time in a month

issue commentsosreport/sos

[4.2] avocado testsuite on Ubuntu

The full buildlog can be found here: https://launchpadlibrarian.net/557777253/buildlog_ubuntu-impish-amd64.sosreport_4.2-1ubuntu2.21_BUILDING.txt.gz

Build-Depends:
 debhelper-compat (= 13),
 avocado,
 dh-python,
 gettext,
 kmod,
 python3-all,
 python3-nose,
 python3-setuptools,
 python3-sphinx,

And I have an autopkgtest restriction to run on a real machine, because sosreport interact may interact with kernel through the kernel plugin.

That's about it.

isolation-machine
The test wants to interact with the kernel, reboot the machine, or other things which fail in a simple schroot and even a container. 
slashdd

comment created time in a month