profile
viewpoint
Robin Jarry rjarry @6WIND Paris, France

CESNET/libyang 200

YANG data modeling language library

sysrepo/sysrepo 175

YANG-based configuration and operational state data store for Unix/Linux applications

gboissinot/gradleplugins 15

Some Gradle plugins

rjarry/rst2word 7

Word writer for docutils (with COM/DCOM interfacing). pywin32 required.

rjarry/rst4eclipse 5

Eclipse plugin for reStructuredText edition.

rjarry/clearquest2rst 2

A clearquest directive for the docutils framework

rjarry/docutils-extensions 2

A docutils extension for allowing easy directive contribution.

rjarry/libyang-cffi 2

Python CFFI bindings to libyang

rjarry/notmuch-gmail 1

Bidirectional sync of Gmail messages with notmuch database

rjarry/ace 0

Ace (Ajax.org Cloud9 Editor)

push event6WIND/libyang

Robin Jarry

commit sha 0facda0f59f416f414feb3280e3190152d61ef4c

doc CHANGE add editorconfig file EditorConfig is a file format and collection of text editor plugins for maintaining consistent coding styles between different editors and IDEs. Initialize the file following the current coding style in existing files. In order for this file to be taken into account (unless they use an editor with built-in EditorConfig support), developers will have to install a plugin. Link: https://editorconfig.org/ Link: https://github.com/editorconfig/editorconfig-emacs Link: https://github.com/editorconfig/editorconfig-vim Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

Robin Jarry

commit sha e0801b07d06c770b82c8b19a7d8511ef57df4d14

bindings CHANGE add new python CFFI bindings This is the integration of https://github.com/rjarry/libyang-cffi In the long term, it is intended to replace the existing SWIG bindings. Some highlights: * Uses CFFI for interaction with libyang.so. * More "pythonic" API. Should be easier to use for Python developers. * All high-level code is in Python. Should ease maintenance. * Virtualenv/pip friendly. * Schema diff feature. * New "print" data format: python dict. To enable the generation, add the -DGEN_PYTHON_CFFI_BINDINGS=ON cmake option. It does not depend on SWIG nor on the CPP bindings. It may be enabled even if -DGEN_LANGUAGE_BINDINGS is OFF. Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

push time in an hour

pull request commentCESNET/libyang

New python bindings

Hi Michal, sure I'll have a look.

rjarry

comment created time in 2 days

pull request commentCESNET/libyang

New python bindings

@rkrejci I have pushed a new version with SWIG bindings left as they are. The default is to build only the legacy bindings. The new CFFI bindings can be enabled (independently from any other SWIG binding) with GEN_PYTHON_CFFI_BINDINGS.

Following this, I have not modified anything in packages since I don't know where you want to go with this.

rjarry

comment created time in 3 days

push event6WIND/libyang

Michal Vasko

commit sha 02b60c298804d5d35515a8fc6a1afecb7ed5d995

build BUGFIX set PIC for compat

view details

Michal Vasko

commit sha 40d9c571903c86354207dff70558d8ca6cb9135b

build BUGFIX do not use CMAKE_C_STANDARD ... because it was added in cmake 3.1.

view details

Michal Vasko

commit sha 6ef875b2201fbe5c690c24d748381a9aca4d7ecf

Merge branch 'master' into devel

view details

Robin Jarry

commit sha f0412f45598051438fe22c8f075f00faf63934de

doc CHANGE add editorconfig file EditorConfig is a file format and collection of text editor plugins for maintaining consistent coding styles between different editors and IDEs. Initialize the file following the current coding style in existing files. In order for this file to be taken into account (unless they use an editor with built-in EditorConfig support), developers will have to install a plugin. Link: https://editorconfig.org/ Link: https://github.com/editorconfig/editorconfig-emacs Link: https://github.com/editorconfig/editorconfig-vim Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

Robin Jarry

commit sha 26173d6736ecd1bc6899255c0b26f7e62d3c5541

bindings CHANGE add new python CFFI bindings This is the integration of https://github.com/rjarry/libyang-cffi In the long term, it is intended to replace the existing SWIG bindings. Some highlights: * Uses CFFI for interaction with libyang.so. * More "pythonic" API. Should be easier to use for Python developers. * All high-level code is in Python. Should ease maintenance. * Virtualenv/pip friendly. * Schema diff feature. * New "print" data format: python dict. To enable the generation, add the -DGEN_PYTHON_CFFI_BINDINGS=ON cmake option. It does not depend on SWIG nor on the CPP bindings. It may be enabled even if -DGEN_LANGUAGE_BINDINGS is OFF. Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

push time in 3 days

pull request commentCESNET/libyang

New python bindings

Hi @rkrejci,

Sure I can leave the SWIG python bindings. However, leaving both bindings enabled by default may be confusing. Building both python bindings should be possible but not the default in my opinion.

The question is: which binding is enabled by default? And with which cmake option?

I am on vacation today, I'll have a closer look tomorrow.

rjarry

comment created time in 4 days

push event6WIND/libyang

Robin Jarry

commit sha 445aa317f0373234108281abbcf24ec064f3d68e

bindings CHANGE rewrite python binding from scratch Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

push time in 5 days

pull request commentCESNET/libyang

New python bindings

Thanks Michal, I rebased on the latest devel and changed the python code accordingly.

rjarry

comment created time in 5 days

push event6WIND/libyang

Michal Vasko

commit sha d3b99126658c8f6ef09c889dfaccd048a9249c7b

build CHANGE include major SO version in plugins dir So that multiple versions can coexist on a system.

view details

Michal Vasko

commit sha 641f0c8069830661758100aafd423f731091dcd1

VERSION bump to version 1.0.169

view details

Michal Vasko

commit sha b17f4e81b921bcbc18cdf079c563ad42cedae5a9

fixup! build CHANGE include major SO version in plugins dir

view details

Michal Vasko

commit sha baa7efdb1a0386d6833f9067b1be4aebbb094295

printer FEATURE function to check if a node should be printed ... added into public API.

view details

Michal Vasko

commit sha c831064cb7589ff7bd0c22d6b2769b685bead102

schema tree FETAURE function for getting data path format string pattern ... for schema nodes. Refs #1087 Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

Michal Vasko

commit sha c7170184e6c3f8a40b692a7f7d252ea0e374dd18

SOVERSION bump to version 1.8.0

view details

Michal Vasko

commit sha 578cb0087701dc5b92089199320f6fe6e0bb9ae2

VERSION bump to version 1.0.170

view details

Robin Jarry

commit sha 6de185be03c7e6a1bf832006869e8720eed2da57

doc CHANGE add editorconfig file EditorConfig is a file format and collection of text editor plugins for maintaining consistent coding styles between different editors and IDEs. Initialize the file following the current coding style in existing files. In order for this file to be taken into account (unless they use an editor with built-in EditorConfig support), developers will have to install a plugin. Link: https://editorconfig.org/ Link: https://github.com/editorconfig/editorconfig-emacs Link: https://github.com/editorconfig/editorconfig-vim Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

Robin Jarry

commit sha a365ad94f2fb11f304f31a2f2a7a6be7076cc402

bindings CHANGE rewrite python binding from scratch Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

push time in 5 days

pull request commentCESNET/libyang

New python bindings

@michalvasko I have made some additional commits to address what I mentioned about new functions in the C library. Please tell me if you prefer that I push them into a separate pull request.

Also, I'd like to support ENABLE_STATIC in the python bindings but there is an link error. I am not sure how to fix it:

x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/tmp/build/python/include -I/usr/include/python3.5m -c /tmp/build/python/temp.linux-x86_64-3.5/_libyang.c -o /tmp/build/python/temp.linux-x86_64-3.5/tmp/build/python/temp.linux-x86_64-3.5/_libyang.o -Werror -std=c99
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 /tmp/build/python/temp.linux-x86_64-3.5/tmp/build/python/temp.linux-x86_64-3.5/_libyang.o -L/tmp/build -lyang -o /tmp/build/python/lib.linux-x86_64-3.5/_libyang.cpython-35m-x86_64-linux-gnu.so
/usr/bin/ld: /tmp/build/libyang.a(common.c.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/tmp/build/libyang.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
rjarry

comment created time in 5 days

push event6WIND/libyang

Robin Jarry

commit sha 50688b486527b4c4ecc48fffb54c65419e86d584

printer CHANGE export lyd_toprint in public symbols Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

Robin Jarry

commit sha ce0358fd51c6f71c5b421695a6f6c1679a49550a

schema CHANGE add new lys_data_path_pattern function Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

Robin Jarry

commit sha bfc2d49b40c4fa9f294e5b9ca229150a6c00055b

bindings CHANGE rewrite python binding from scratch Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

push time in 5 days

pull request commentCESNET/libyang

New python bindings

@michalvasko would that be OK to export ly_toprint() in the public symbols for libyang.so?

Also, would that be OK to add one utility function:

/**
 * @brief Return the data path with list key placeholders.
 *
 * @param[in] node Schema node for which to get the data path.
 * @param[in] placeholder String that will be inserted in place of the list key values.
 * @return NULL on error, on success the buffer for the resulting path is
*          allocated and caller is supposed to free it with free().
 */
char *lys_data_path_pattern(const struct lys_node *node, const char *placeholder);

Example of use for the Python bindings:

p = lib.lys_data_path_pattern(self._snode, '%s')
p = ffi.string(p).decode('utf-8')
# /module1:container/module2:list[key1='%s']/sublist[key2='%s']/leaf
p %= ('val1', 'val2')
# /module1:container/module2:list[key1='val1']/sublist[key2='val2']/leaf
rjarry

comment created time in 6 days

pull request commentCESNET/libyang

New python bindings

So source.c contains all the internal functions you needed directly in the bindings?

Yes. Most of them are trivial and could be implemented directly in python. However, for the sake of performance, I did some of the work in C. Maybe there are some that could be integrated in the C library:

https://github.com/CESNET/libyang/pull/1087/files#diff-8b2dff95a233a8f66dea33417af39c83R9 https://github.com/CESNET/libyang/pull/1087/files#diff-8b2dff95a233a8f66dea33417af39c83R43 https://github.com/CESNET/libyang/pull/1087/files#diff-8b2dff95a233a8f66dea33417af39c83R105

The most critical one (I guess) is ly_toprint() which is mandatory for the python dict data format. Would it be possible to export this function in the API? Or maybe another version of it.

https://github.com/CESNET/libyang/pull/1087/files#diff-8b2dff95a233a8f66dea33417af39c83R251 https://github.com/CESNET/libyang/pull/1087/files#diff-8b2dff95a233a8f66dea33417af39c83R131 https://github.com/CESNET/libyang/pull/1087/files#diff-bb57130e2bf44e477f946061dda45891R235

rjarry

comment created time in 7 days

push event6WIND/libyang

Robin Jarry

commit sha 30e1d819c612d673585e693c9cef43581567f2fd

fixup! bindings CHANGE rewrite python binding from scratch

view details

push time in 7 days

Pull request review commentCESNET/libyang

New python bindings

+message(STATUS "Python version ${GEN_PYTHON_VERSION} was selected")+unset(PYTHON_LIBRARY CACHE)+unset(PYTHON_EXECUTABLE CACHE)+unset(PYTHON_INCLUDE_DIR CACHE)++if(${GEN_PYTHON_VERSION} STREQUAL "2")+    find_package(PythonInterp 2 REQUIRED)+    find_package(PythonLibs 2 REQUIRED)+    if(NOT PYTHONLIBS_FOUND)+        message(WARNING "Did not found Python version 2.x")+        message(STATUS "Libyang supports Python 2.x and Python 3.x")+    endif()

Yes since https://github.com/rjarry/libyang-cffi supports it. However, since Python 2 is officially EOL since the begining of 2020, I was not sure if you wanted to keep official Python 2 support.

rjarry

comment created time in 7 days

Pull request review commentCESNET/libyang

New python bindings

 configure_file(${PROJECT_SOURCE_DIR}/packages/debian.${CPP_PACKAGE}.install                ${PROJECT_BINARY_DIR}/build-packages/debian.${CPP_PACKAGE}.install COPYONLY) configure_file(${PROJECT_SOURCE_DIR}/packages/debian.${CPP_PACKAGE}-dev.install                ${PROJECT_BINARY_DIR}/build-packages/debian.${CPP_PACKAGE}-dev.install COPYONLY)-# no python package for Debian because there is only SWIG 3.10 on Debian 9 :-/-#configure_file(${PROJECT_SOURCE_DIR}/packages/debian.${PYTHON_PACKAGE}.install-#               ${PROJECT_BINARY_DIR}/build-packages/debian.${PYTHON_PACKAGE}.install COPYONLY)+configure_file(${PROJECT_SOURCE_DIR}/packages/debian.${PYTHON_PACKAGE}.install+               ${PROJECT_BINARY_DIR}/build-packages/debian.${PYTHON_PACKAGE}.install COPYONLY)

Yes I guess CFFI has been present in debian/ubuntu for a long time.

https://packages.debian.org/stretch/python3-cffi https://packages.ubuntu.com/xenial/python3-cffi

The version is a bit old but it "should" work.

rjarry

comment created time in 7 days

Pull request review commentCESNET/libyang

New python bindings

 set(PACKAGE "libyang") set(CPP_PACKAGE "libyang-cpp")-set(PYTHON_PACKAGE "python3-yang")+set(PYTHON_PACAKGE "python3-libyang")

I meant "confusing" because the python package name is libyang.

rjarry

comment created time in 7 days

Pull request review commentCESNET/libyang

New python bindings

 set(PACKAGE "libyang") set(CPP_PACKAGE "libyang-cpp")-set(PYTHON_PACKAGE "python3-yang")+set(PYTHON_PACAKGE "python3-libyang")

Just saw the typo :)

About the name, I'm not sure. I find it confusing. I'll revert the previous name.

rjarry

comment created time in 7 days

push event6WIND/libyang

Robin Jarry

commit sha b534425a3825d670100e956897eec616ea4706c8

fixup! bindings CHANGE rewrite python binding from scratch

view details

push time in 7 days

push event6WIND/libyang

Robin Jarry

commit sha bcb4543950f57f7a37ef362c8ba3033076b0d48f

fixup! bindings CHANGE rewrite python binding from scratch

view details

push time in 7 days

push event6WIND/libyang

Robin Jarry

commit sha 50082f1c44237c780b833b5a56738372f6e63792

fixup! bindings CHANGE rewrite python binding from scratch

view details

push time in 7 days

pull request commentCESNET/libyang

New python bindings

As I expected, the static build does not work...

I'll disable it for the time being.

rjarry

comment created time in 7 days

push event6WIND/libyang

Robin Jarry

commit sha 393bf4f6efc4589300685756db7141c8c4539f53

fixup! bindings CHANGE rewrite python binding from scratch

view details

push time in 7 days

push event6WIND/libyang

Robin Jarry

commit sha c522498be36672a2163f86fae560b2ef0ae472a4

fixup! bindings CHANGE rewrite python binding from scratch

view details

push time in 7 days

push event6WIND/libyang

Robin Jarry

commit sha b03a9aabb94ef8e6a639ca2f5afab5818c8894cd

fixup! bindings CHANGE rewrite python binding from scratch

view details

push time in 7 days

push event6WIND/libyang

Robin Jarry

commit sha 952ff63d609b407dfdd53393b50b79537ee52d3a

bindings CHANGE rewrite python binding from scratch Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

push time in 7 days

Pull request review commentCESNET/libyang

New python bindings

+/*+ * Copyright (c) 2018-2019 Robin Jarry+ * SPDX-License-Identifier: BSD-3-Clause+ */++#include <libyang/libyang.h>++static const struct lys_ext_instance *+lypy_find_ext(const struct lys_ext_instance **ext, uint8_t ext_size,+	const char *name, const char *prefix, const char *arg_value)+{+	const struct lys_ext_instance *inst;+	const struct lys_module *module;+	const struct lys_ext *def;+	uint8_t i;++	if (!ext)+		goto notfound;++	for (i = 0; i < ext_size; i++) {+		inst = ext[i];+		def = inst->def;+		if (name && strcmp(def->name, name) != 0)+			continue;+		if (prefix) {+			module = lys_main_module(def->module);+			if (!module)+				goto notfound;+			if (strcmp(module->name, prefix) != 0)+				continue;+		}+		if (arg_value && inst->arg_value) {+			if (strcmp(arg_value, inst->arg_value) != 0)+				continue;+		}+		return inst;+	}++notfound:+	return NULL;+}++static char *lypy_data_path_pattern(const struct lys_node *node)+{+	const struct lys_module *prev_mod, *mod;+	char *xpath = NULL, *keys = NULL;+	struct ly_set *set = NULL;;+	size_t x;++	if (!node)+		goto cleanup;++	set = ly_set_new();+	if (!set)+		goto cleanup;++	while (node) {+		ly_set_add(set, (void *)node, 0);+		do {+			node = lys_parent(node);+		} while (node && !(node->nodetype & (+			LYS_CONTAINER | LYS_LIST | LYS_RPC)));+	}++	xpath = malloc(2048);+	if (!xpath)+		goto cleanup;+	keys = malloc(512);+	if (!keys)+		goto cleanup;++	x = 0;+	xpath[0] = '\0';++	prev_mod = NULL;+	for (int i = set->number - 1; i > -1; --i) {+		size_t k = 0;+		keys[0] = '\0';+		node = set->set.s[i];+		if (node->nodetype == LYS_LIST) {+			const struct lys_node_list *list;+			list = (const struct lys_node_list *)node;+			for (uint8_t j = 0; j < list->keys_size; j++) {+				k += sprintf(keys + k, "[%s='%%s']",+					list->keys[j]->name);+			}+		}++		mod = lys_node_module(node);+		if (mod && mod != prev_mod) {+			prev_mod = mod;+			x += sprintf(xpath + x, "/%s:%s%s",+				mod->name, node->name, keys);+		} else {+			x += sprintf(xpath + x, "/%s%s", node->name, keys);+		}+	}++cleanup:+	ly_set_free(set);+	free(keys);+	return xpath;+}++static char *lypy_node_fullname(const struct lys_node *node)+{+	const struct lys_module *module;+	char *fullname = NULL;++	module = lys_node_module(node);+	if (!module)+		return NULL;++	if (asprintf(&fullname, "%s:%s", module->name, node->name) < 0)+		return NULL;++	return fullname;+}++static LY_ERR lypy_get_errno(void)+{+	return ly_errno;+}++static void lypy_set_errno(LY_ERR err)+{+	ly_errno = err;+}++static int+lyd_wd_toprint(const struct lyd_node *node, int options)+{+    const struct lyd_node *subroot, *next, *elem;+    int flag = 0;++    if (options & LYP_WD_TRIM) {+        /* do not print default nodes */+        if (node->dflt) {+            /* implicit default node */+            return 0;+        } else if (node->schema->nodetype & (LYS_LEAF | LYS_LEAFLIST)) {+            if (lyd_wd_default((struct lyd_node_leaf_list *)node)) {+                /* explicit default node */+                return 0;+            }+        } else if ((node->schema->nodetype & (LYS_CONTAINER)) && !((struct lys_node_container *)node->schema)->presence) {+            /* get know if non-presence container contains non-default node */+            for (subroot = node->child; subroot && !flag; subroot = subroot->next) {+                LY_TREE_DFS_BEGIN(subroot, next, elem) {+                    if (elem->dflt) {+                        /* skip subtree */+                        goto trim_dfs_nextsibling;+                    }+                    switch (elem->schema->nodetype) {+                    case LYS_LEAF:+                    case LYS_LEAFLIST:+                        if (!lyd_wd_default((struct lyd_node_leaf_list *)elem)) {+                            /* non-default node */+                            flag = 1;+                        }+                        break;+                    case LYS_ANYDATA:+                    case LYS_ANYXML:+                    case LYS_NOTIF:+                    case LYS_ACTION:+                    case LYS_LIST:+                        /* non-default nodes */+                        flag = 1;+                        break;+                    case LYS_CONTAINER:+                        if (((struct lys_node_container *)elem->schema)->presence) {+                            /* non-default node */+                            flag = 1;+                        }+                        break;+                    default:+                        break;+                    }+                    if (flag) {+                        break;+                    }++                    /* modified LY_TREE_DFS_END */+                    /* select element for the next run - children first */+                    /* child exception for leafs, leaflists and anyxml without children */+                    if (elem->schema->nodetype & (LYS_LEAF | LYS_LEAFLIST | LYS_ANYDATA)) {+                        next = NULL;+                    } else {+                        next = elem->child;+                    }+                    if (!next) {+trim_dfs_nextsibling:+                        /* no children */+                        if (elem == subroot) {+                            /* we are done, (START) has no children */+                            break;+                        }+                        /* try siblings */+                        next = elem->next;+                    }+                    while (!next) {+                        /* parent is already processed, go to its sibling */+                        elem = elem->parent;+                        /* no siblings, go back through parents */+                        if (elem->parent == subroot->parent) {+                            /* we are done, no next element to process */+                            break;+                        }+                        next = elem->next;+                    }+                }+            }+            if (!flag) {+                /* only default nodes in subtree, do not print the container */+                return 0;+            }+        }+    } else if (node->dflt && !(options & LYP_WD_MASK) && !(node->schema->flags & LYS_CONFIG_R)) {+        /* LYP_WD_EXPLICIT+         * - print only if it contains status data in its subtree */+        LY_TREE_DFS_BEGIN(node, next, elem) {+            if (elem->schema->flags & LYS_CONFIG_R) {+                flag = 1;+                break;+            }+            LY_TREE_DFS_END(node, next, elem)+        }+        if (!flag) {+            return 0;+        }+    } else if (node->dflt && node->schema->nodetype == LYS_CONTAINER && !(options & LYP_KEEPEMPTYCONT)) {+        /* avoid empty default containers */+        LY_TREE_DFS_BEGIN(node, next, elem) {+            if (elem->schema->nodetype != LYS_CONTAINER) {+                flag = 1;+                break;+            }+            LY_TREE_DFS_END(node, next, elem)+        }+        if (!flag) {+            return 0;+        }+    }++    return 1;+}++/* This function is not exported in the public symbols of libyang.so.+ * It has been copied here verbatim. */+int+lyd_toprint(const struct lyd_node *node, int options)+{+    struct lys_node *scase, *sparent;+    struct lyd_node *first;++    if (!lyd_wd_toprint(node, options)) {+        /* wd says do not print, but make exception for direct descendants of case nodes without other printable nodes */+        for (sparent = lys_parent(node->schema); sparent && (sparent->nodetype == LYS_USES); sparent = lys_parent(sparent));+        if (!sparent || (sparent->nodetype != LYS_CASE)) {+            /* parent not a case */+            return 0;+        }+        scase = sparent;++        for (sparent = lys_parent(scase); sparent && (sparent->nodetype == LYS_USES); sparent = lys_parent(sparent));+        if (!sparent || (sparent->nodetype != LYS_CHOICE)) {+            /* weird */+            return 0;+        }+        if (((struct lys_node_choice *)sparent)->dflt == scase) {+            /* this is a default case, respect the previous original toprint flag */+            return 0;+        }++        /* try to find a sibling that will be printed */+        for (first = node->prev; first->prev->next; first = first->prev);+        LY_TREE_FOR(first, first) {+            if (first == node) {+                /* skip this node */+                continue;+            }++            /* find schema parent, whether it is the same case */+            for (sparent = lys_parent(first->schema); sparent && (sparent->nodetype == LYS_USES); sparent = lys_parent(sparent));+            if ((sparent == scase) && lyd_wd_toprint(first, options)) {+                /* this other node will be printed, we do not have to print the current one */+                return 0;+            }+        }++        /* there is no case child that will be printed, print this node */+        return 1;+    }++    return 1;+}

As I said, there is some non-API C code that was needed into the python binding. I copied it since I had no way to call it directly. However, it would be best to leave this code into libyang.so and make it available in the API somehow.

rjarry

comment created time in 7 days

Pull request review commentCESNET/libyang

New python bindings

+/*+ * Copyright (c) 2018-2019 Robin Jarry+ * SPDX-License-Identifier: BSD-3-Clause+ */++#include <libyang/libyang.h>++static const struct lys_ext_instance *+lypy_find_ext(const struct lys_ext_instance **ext, uint8_t ext_size,+	const char *name, const char *prefix, const char *arg_value)+{+	const struct lys_ext_instance *inst;+	const struct lys_module *module;+	const struct lys_ext *def;+	uint8_t i;++	if (!ext)+		goto notfound;++	for (i = 0; i < ext_size; i++) {+		inst = ext[i];+		def = inst->def;+		if (name && strcmp(def->name, name) != 0)+			continue;+		if (prefix) {+			module = lys_main_module(def->module);+			if (!module)+				goto notfound;+			if (strcmp(module->name, prefix) != 0)+				continue;+		}+		if (arg_value && inst->arg_value) {+			if (strcmp(arg_value, inst->arg_value) != 0)+				continue;+		}+		return inst;+	}++notfound:+	return NULL;+}++static char *lypy_data_path_pattern(const struct lys_node *node)+{+	const struct lys_module *prev_mod, *mod;+	char *xpath = NULL, *keys = NULL;+	struct ly_set *set = NULL;;+	size_t x;++	if (!node)+		goto cleanup;++	set = ly_set_new();+	if (!set)+		goto cleanup;++	while (node) {+		ly_set_add(set, (void *)node, 0);+		do {+			node = lys_parent(node);+		} while (node && !(node->nodetype & (+			LYS_CONTAINER | LYS_LIST | LYS_RPC)));+	}++	xpath = malloc(2048);+	if (!xpath)+		goto cleanup;+	keys = malloc(512);+	if (!keys)+		goto cleanup;++	x = 0;+	xpath[0] = '\0';++	prev_mod = NULL;+	for (int i = set->number - 1; i > -1; --i) {+		size_t k = 0;+		keys[0] = '\0';+		node = set->set.s[i];+		if (node->nodetype == LYS_LIST) {+			const struct lys_node_list *list;+			list = (const struct lys_node_list *)node;+			for (uint8_t j = 0; j < list->keys_size; j++) {+				k += sprintf(keys + k, "[%s='%%s']",+					list->keys[j]->name);+			}+		}++		mod = lys_node_module(node);+		if (mod && mod != prev_mod) {+			prev_mod = mod;+			x += sprintf(xpath + x, "/%s:%s%s",+				mod->name, node->name, keys);+		} else {+			x += sprintf(xpath + x, "/%s%s", node->name, keys);+		}+	}++cleanup:+	ly_set_free(set);+	free(keys);+	return xpath;+}++static char *lypy_node_fullname(const struct lys_node *node)+{+	const struct lys_module *module;+	char *fullname = NULL;++	module = lys_node_module(node);+	if (!module)+		return NULL;++	if (asprintf(&fullname, "%s:%s", module->name, node->name) < 0)+		return NULL;++	return fullname;+}++static LY_ERR lypy_get_errno(void)+{+	return ly_errno;+}++static void lypy_set_errno(LY_ERR err)+{+	ly_errno = err;+}++static int

As I said, there is some non-API C code that was needed into the python binding. I copied it since I had no way to call it directly. However, it would be best to leave this code into libyang.so and make it available in the API somehow.

rjarry

comment created time in 7 days

PR opened CESNET/libyang

New python bindings

Hi @michalvasko @rkrejci @Dajvid,

As promised, here is a first draft of the integration of https://github.com/rjarry/libyang-cffi.

Few notes:

  • I tried to fit in with cmake as much as possible but I am not very proficient with it. There may be atrocities to fix.
  • Python distutils/setuptools command line interface is horrible. The only "sane" way to ensure that arguments are taken into account is to use a custom "user configuration file" (generated .pydistutils.cfg) and force a value to HOME during the build so that distutils/setuptools load it. This approach was borrowed from Piotr Ożarowski's pybuild which is used extensively in Debian and Ubuntu to generate Python .deb packages.

What is missing:

  • Write actual examples
  • More unit tests?
  • API functions not exported
  • Python doc strings
  • Integration in the packages subfolder (for .deb/.rpm and pypi.org)

What do you think?

+4252 -2452

0 comment

44 changed files

pr created time in 7 days

create barnch6WIND/libyang

branch : python

created branch time in 7 days

issue commentCESNET/netopeer2

Ubuntu 20.04 LTS installation problem

Hi Michal,

Are you encountering this into a linux container? If so, this may be related to the default docker AppArmor profile.

I already encountered this kind of confusing errors because of AppArmor.

michalvasko

comment created time in 7 days

delete branch 6WIND/libnetconf2

delete branch : io-pollup

delete time in 10 days

pull request commentCESNET/libnetconf2

io BUGFIX do not fail when there is data left to read

Thanks Michal :+1:

rjarry

comment created time in 10 days

PR opened CESNET/libnetconf2

io BUGFIX do not fail when there is data left to read

When using the UNIX transport and depending on the timing between the server and client, the client may read the final reply to the <close-session> rpc after the server has closed the socket. When that occurs, poll() returns POLLHUP in revents meaning the server closed the socket which causes the following error on the client:

ERR: Session 1: communication channel unexpectedly closed.
ERR: Session 1: failed to receive a reply to <close-session>.

When POLLHUP is returned in revents, make sure POLLIN is not present as well (if it is, it means that there is some data left to be read in the socket buffer).

See https://www.greenend.org.uk/rjk/tech/poll.html for more details.

+8 -6

0 comment

1 changed file

pr created time in 10 days

create barnch6WIND/libnetconf2

branch : io-pollup

created branch time in 10 days

pull request commentmartinblech/xmltodict

Fix UnicodeDecodeError with python3 and no utf-8 locale installed

The travis build failed for jython: https://travis-ci.com/github/martinblech/xmltodict/jobs/333716738

Searching for nose
Reading https://pypi.python.org/simple/nose/
Download error on https://pypi.python.org/simple/nose/: [Errno 1] Tag mismatch! (javax.crypto.AEADBadTagException: Tag mismatch!) -- Some packages may not be found!
Couldn't find index page for 'nose' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
No local packages or download links found for nose
error: Could not find suitable distribution for Requirement.parse('nose')
The command "$HOME/jython/bin/easy_install nose" failed and exited with 1 during .

I'm not sure this is related to this patch.

rjarry

comment created time in 11 days

PR opened martinblech/xmltodict

setup: open README.md with a specific encoding

With python3, the built-in open() function opens files in "text" mode by default (i.e. the f.read() method reads bytes and tries to decode them to unicode using the interpreter default encoding). Depending on the system locale, this default encoding may be 'ascii'. Which leads to the following error:

~$ python3 setup.py egg_info
Traceback (most recent call last):
  File "setup.py", line 12, in <module>
    long_description = f.read()
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2587: ordinal not in range(128)

The character at position 2587 is ô (from Fantômas) and is utf-8 encoded.

Force to open the file in "binary" mode (which is the only mode on python2) and explicitly decode the read bytes to unicode using the 'utf-8' codec.

Fixes: 8374e7d84f92 ("Use Markdown long_description on PyPI (#190)")

+2 -2

0 comment

1 changed file

pr created time in 11 days

create barnchrjarry/xmltodict

branch : py3-decode-error

created branch time in 11 days

push eventrjarry/xmltodict

push time in 11 days

push eventrjarry/xmltodict

Robin Jarry

commit sha a2325c96ba2ec4af4472c7398edf65c94634f96e

setup: open README.md with a specific encoding With python3, the built-in open() function opens files in "text" mode by default (i.e. the f.read() method reads bytes and tries to decode them to unicode using the interpreter default encoding). Depending on the system locale, this default encoding may be 'ascii'. Which leads to the following error: ~$ python3 setup.py egg_info Traceback (most recent call last): File "setup.py", line 12, in <module> long_description = f.read() File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2587: ordinal not in range(128) The character at position 2587 is 'ô' (from Fantômas) and is utf-8 encoded. Force to open the file in "binary" mode (which is the only mode on python2) and explicitly decode the read bytes to unicode using the 'utf-8' codec. Fixes: 8374e7d84f92 ("Use Markdown long_description on PyPI (#190)") Signed-off-by: Robin Jarry <robin.jarry@6wind.com>

view details

push time in 11 days

fork rjarry/xmltodict

Python module that makes working with XML feel like you are working with JSON

fork in 11 days

issue closedsysrepo/sysrepo

Config change callbacks not always called

Hi Michal,

sorry, this issue affects Netopeer2 as well but I have the feeling the problem is in sysrepo.

I have built and installed the following versions (with -DCMAKE_BUILD_TYPE=release):

https://github.com/CESNET/libyang/commit/b11c70c5246da3be3e153e5919a54e9c84bd8647 https://github.com/CESNET/libnetconf2/commit/b0f5e0ea0d9adf18827ab04967f90cc100464362 https://github.com/sysrepo/sysrepo/commit/a9e47b7ac54b9817a558020c27ac82b767ff3b95 https://github.com/CESNET/Netopeer2/commit/83012918f76a966591bbdce4c27277c86aade469

I have applied the following debug changes on sysrepo before building:

diff --git a/src/shm_sub.c b/src/shm_sub.c
index d33b25e910f0..da2ad0212638 100644
--- a/src/shm_sub.c
+++ b/src/shm_sub.c
@@ -980,13 +980,16 @@ sr_shmsub_change_notify_change_done(struct sr_mod_info_s *mod_info, sr_sid_t sid
     sr_shm_t shm_sub = SR_SHM_INITIALIZER;
 
     while ((mod = sr_modinfo_next_mod(mod, mod_info, mod_info->diff, &aux))) {
+        printf("sr_shmsub_change_notify_change_done: module=%s\n", mod->ly_mod->name);
         /* first check that there actually are some value changes (and not only dflt changes) */
         if (!sr_shmsub_change_notify_diff_has_changes(mod, mod_info->diff)) {
+            printf("sr_shmsub_change_notify_change_done: module=%s no changes\n", mod->ly_mod->name);
             continue;
         }
 
         if (!sr_shmsub_change_notify_has_subscription(mod_info->conn->ext_shm.addr, mod, mod_info->ds, SR_SUB_EV_DONE,
                 &cur_priority)) {
+            printf("sr_shmsub_change_notify_change_done: module=%s no subs\n", mod->ly_mod->name);
             /* no subscriptions interested in this event */
             continue;
         }
@@ -1031,6 +1034,17 @@ sr_shmsub_change_notify_change_done(struct sr_mod_info_s *mod_info, sr_sid_t sid
                     subscriber_count, 0, diff_lyb, diff_lyb_len, mod->ly_mod->name);
 
             /* notify using event pipe and do not wait for subscribers */
+            printf("==========================\n");
+            printf("sr_shmsub_change_notify_change_done: module=%s new configuration\n", mod->ly_mod->name);
+            const struct lyd_node *mod_data;
+            LY_TREE_FOR(mod_info->data, mod_data) {
+                if (!strcmp(lyd_node_module(mod_data)->name, mod->ly_mod->name)) {
+                    lyd_print_file(stdout, mod_data, LYD_XML, LYP_FORMAT);
+                    break;
+                }
+            }
+            printf("sr_shmsub_change_notify_change_done: module=%s notify subscribers\n", mod->ly_mod->name);
+            printf("==========================\n");
             if ((err_info = sr_shmsub_change_notify_evpipe(mod_info->conn->ext_shm.addr, mod, mod_info->ds,
                     SR_SUB_EV_DONE, cur_priority))) {
                 goto cleanup_wrunlock;

And the following debug changes to Netopeer2:

diff --git a/server/netconf_server.c b/server/netconf_server.c
index 64d200e8f667..641d34082cdc 100644
--- a/server/netconf_server.c
+++ b/server/netconf_server.c
@@ -188,6 +188,17 @@ np2srv_endpt_tcp_params_cb(sr_session_ctx_t *session, const char *UNUSED(module_
         /* find name */
         endpt_name = ((struct lyd_node_leaf_list *)node->parent->parent->parent->child)->value_str;
 
+#define CHANGE_NAME(c) ( \
+    (c == SR_OP_CREATED) ? "CREATED" : ( \
+        (c == SR_OP_MODIFIED) ? "DELETED" : ( \
+            (c == SR_OP_DELETED) ? "DELETED" : "???")))
+
+        char *dpath = lyd_path(node);
+        ERR("np2srv_endpt_tcp_params_cb %s: %s", CHANGE_NAME(op), dpath);
+        free(dpath);
+
+#undef CHANGE_NAME
+
         if (!strcmp(node->schema->name, "local-address")) {
             if ((op == SR_OP_CREATED) || (op == SR_OP_MODIFIED)) {
                 if (nc_server_endpt_set_address(endpt_name, ((struct lyd_node_leaf_list *)node)->value_str)) {
diff --git a/server/netconf_server_ssh.c b/server/netconf_server_ssh.c
index e61b3c254aca..2e9e27c02d72 100644
--- a/server/netconf_server_ssh.c
+++ b/server/netconf_server_ssh.c
@@ -281,15 +281,19 @@ np2srv_endpt_ssh_cb(sr_session_ctx_t *session, const char *UNUSED(module_name),
     while ((rc = sr_get_change_tree_next(session, iter, &op, &node, &prev_val, &prev_list, &prev_dflt)) == SR_ERR_OK) 
         /* get name */
         endpt_name = ((struct lyd_node_leaf_list *)node->parent->child)->value_str;
+        char *dpath = lyd_path(node);
 
         /* ignore other operations */
         if (op == SR_OP_CREATED) {
+            ERR("np2srv_endpt_ssh_cb CREATED: %s", dpath);
             rc = nc_server_add_endpt(endpt_name, NC_TI_LIBSSH);
             /* turn off all auth methods by default */
             nc_server_ssh_endpt_set_auth_methods(endpt_name, 0);
         } else if (op == SR_OP_DELETED) {
+            ERR("np2srv_endpt_ssh_cb DELETED: %s", dpath);
             rc = nc_server_del_endpt(endpt_name, NC_TI_LIBSSH);
         }
+        free(dpath);
         if (rc) {
             sr_free_change_iter(iter);
             return SR_ERR_INTERNAL;

When starting netopeer2-server the startup configuration is applied as expected:

user@tom:/tmp/netopeer$ sudo netopeer2-server -d &
[1] 2498
[ERR]: NP: np2srv_endpt_ssh_cb CREATED: /ietf-netconf-server:netconf-server/listen/endpoint[name='default-ssh']/ssh
[ERR]: NP: np2srv_endpt_tcp_params_cb CREATED: /ietf-netconf-server:netconf-server/listen/endpoint[name='default-ssh']/ssh/tcp-server-parameters/local-address
[ERR]: NP: np2srv_endpt_tcp_params_cb CREATED: /ietf-netconf-server:netconf-server/listen/endpoint[name='default-ssh']/ssh/tcp-server-parameters/keepalives
[ERR]: NP: np2srv_endpt_tcp_params_cb CREATED: /ietf-netconf-server:netconf-server/listen/endpoint[name='default-ssh']/ssh/tcp-server-parameters/local-port

And the server is listening as configured:

user@tom:/tmp/netopeer$ sudo ss -nlpt | grep netopeer
LISTEN   0   5    0.0.0.0:830      0.0.0.0:*      users:(("netopeer2-serve",pid=2499,fd=25))

However, when replacing the configuration by something different, the config change callbacks are not called (well, in some rare cases, they are called, but I did not manage to understand in which conditions).

user@tom:/tmp/netopeer$ sysrepocfg -m ietf-netconf-server -f xml -Xdefault.xml
user@tom:/tmp/netopeer$ cp default.xml two.xml
user@tom:/tmp/netopeer$ vim two.xml 
user@tom:/tmp/netopeer$ sysrepocfg -m ietf-netconf-server -f xml -Itwo.xml
sr_shmsub_change_notify_change_done: module=ietf-netconf-server
==========================
sr_shmsub_change_notify_change_done: module=ietf-netconf-server new configuration
<netconf-server xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-server">
  <listen>
    <endpoint>
      <name>ssh-default-port</name>
      <ssh>
        <tcp-server-parameters>
          <local-address>0.0.0.0</local-address>
          <keepalives>
            <idle-time>1</idle-time>
            <max-probes>10</max-probes>
            <probe-interval>5</probe-interval>
          </keepalives>
        </tcp-server-parameters>
        <ssh-server-parameters>
          <server-identity>
            <host-key>
              <name>default-key</name>
              <public-key>
                <keystore-reference>genkey</keystore-reference>
              </public-key>
            </host-key>
          </server-identity>
          <client-authentication>
            <supported-authentication-methods>
              <publickey/>
              <passsword/>
              <other>interactive</other>
            </supported-authentication-methods>
            <users/>
          </client-authentication>
        </ssh-server-parameters>
      </ssh>
    </endpoint>
    <endpoint>
      <name>ssh-ipv6-port-12345</name>
      <ssh>
        <tcp-server-parameters>
          <local-address>::</local-address>
          <local-port>12345</local-port>
          <keepalives>
            <idle-time>1</idle-time>
            <max-probes>10</max-probes>
            <probe-interval>5</probe-interval>
          </keepalives>
        </tcp-server-parameters>
        <ssh-server-parameters>
          <server-identity>
            <host-key>
              <name>default-key</name>
              <public-key>
                <keystore-reference>genkey</keystore-reference>
              </public-key>
            </host-key>
          </server-identity>
          <client-authentication>
            <supported-authentication-methods>
              <publickey/>
              <passsword/>
              <other>interactive</other>
            </supported-authentication-methods>
            <users/>
          </client-authentication>
        </ssh-server-parameters>
      </ssh>
    </endpoint>
  </listen>
</netconf-server>
sr_shmsub_change_notify_change_done: module=ietf-netconf-server notify subscribers
==========================
<<<<< NO LOG FROM NETOPEER2-SERVER >>>>
user@tom:/tmp/netopeer$ sudo ss -nlpt | grep netopeer
LISTEN   0   5    0.0.0.0:830      0.0.0.0:*      users:(("netopeer2-serve",pid=2499,fd=25))
<<<<< NO CHANGE IN SOCKETS >>>>>

Also, when applying an empty configuration:

user@tom:/tmp/netopeer$ vim empty.xml 
user@tom:/tmp/netopeer$ cat empty.xml 
<netconf-server xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-server">
</netconf-server>
user@tom:/tmp/netopeer$ sysrepocfg -m ietf-netconf-server -f xml -Iempty.xml
sr_shmsub_change_notify_change_done: module=ietf-netconf-server
==========================
sr_shmsub_change_notify_change_done: module=ietf-netconf-server new configuration
sr_shmsub_change_notify_change_done: module=ietf-netconf-server notify subscribers
==========================
<<<<< NO LOG FROM NETOPEER2-SERVER >>>>
user@tom:/tmp/netopeer$ sudo ss -nlpt | grep netopeer
LISTEN   0   5    0.0.0.0:830      0.0.0.0:*      users:(("netopeer2-serve",pid=2499,fd=25))
<<<<< NO CHANGE IN SOCKETS >>>>>

I don't know if I am doing something wrong (maybe the versions I am using are not well "aligned" with each other).

Thanks in advance for your help!

closed time in 18 days

rjarry

issue commentsysrepo/sysrepo

Config change callbacks not always called

Thanks Michal! It works :)

rjarry

comment created time in 18 days

created tagrjarry/libyang-cffi

tagv1.0.167

Python CFFI bindings to libyang

created time in 18 days

push eventrjarry/libyang-cffi

Robin Jarry

commit sha 1b4f76b121094fa04d0c39f5cc705e4a02a47fdf

data: add options to create_data_path Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 3fc33827783021e5886245643105bd79694e9504

data: factorize code in dict_to_dnode Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha a8d6b8a1b19fe4b27198c581b71feeb455a096e1

dict_to_dnode: free allocated nodes on error Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha f69afe464fe8936464e2b9e7513acf612b1b94f4

Update libyang C lib to version 1.0.167 Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

push time in 18 days

pull request commenthakimel/reveal.js

reveal.js 4.0.0

@hakimel thanks it works now!

hakimel

comment created time in 18 days

issue openedsysrepo/sysrepo

Config change callbacks not always called

Hi Michal,

sorry, this issue affects Netopeer2 as well but I have the feeling the problem is in sysrepo.

I have built and installed the following versions (with -DCMAKE_BUILD_TYPE=release):

https://github.com/CESNET/libyang/commit/b11c70c5246da3be3e153e5919a54e9c84bd8647 https://github.com/CESNET/libnetconf2/commit/b0f5e0ea0d9adf18827ab04967f90cc100464362 https://github.com/sysrepo/sysrepo/commit/a9e47b7ac54b9817a558020c27ac82b767ff3b95 https://github.com/CESNET/Netopeer2/commit/83012918f76a966591bbdce4c27277c86aade469

I have applied the following debug changes on sysrepo before building:

diff --git a/src/shm_sub.c b/src/shm_sub.c
index d33b25e910f0..da2ad0212638 100644
--- a/src/shm_sub.c
+++ b/src/shm_sub.c
@@ -980,13 +980,16 @@ sr_shmsub_change_notify_change_done(struct sr_mod_info_s *mod_info, sr_sid_t sid
     sr_shm_t shm_sub = SR_SHM_INITIALIZER;
 
     while ((mod = sr_modinfo_next_mod(mod, mod_info, mod_info->diff, &aux))) {
+        printf("sr_shmsub_change_notify_change_done: module=%s\n", mod->ly_mod->name);
         /* first check that there actually are some value changes (and not only dflt changes) */
         if (!sr_shmsub_change_notify_diff_has_changes(mod, mod_info->diff)) {
+            printf("sr_shmsub_change_notify_change_done: module=%s no changes\n", mod->ly_mod->name);
             continue;
         }
 
         if (!sr_shmsub_change_notify_has_subscription(mod_info->conn->ext_shm.addr, mod, mod_info->ds, SR_SUB_EV_DONE,
                 &cur_priority)) {
+            printf("sr_shmsub_change_notify_change_done: module=%s no subs\n", mod->ly_mod->name);
             /* no subscriptions interested in this event */
             continue;
         }
@@ -1031,6 +1034,17 @@ sr_shmsub_change_notify_change_done(struct sr_mod_info_s *mod_info, sr_sid_t sid
                     subscriber_count, 0, diff_lyb, diff_lyb_len, mod->ly_mod->name);
 
             /* notify using event pipe and do not wait for subscribers */
+            printf("==========================\n");
+            printf("sr_shmsub_change_notify_change_done: module=%s new configuration\n", mod->ly_mod->name);
+            const struct lyd_node *mod_data;
+            LY_TREE_FOR(mod_info->data, mod_data) {
+                if (!strcmp(lyd_node_module(mod_data)->name, mod->ly_mod->name)) {
+                    lyd_print_file(stdout, mod_data, LYD_XML, LYP_FORMAT);
+                    break;
+                }
+            }
+            printf("sr_shmsub_change_notify_change_done: module=%s notify subscribers\n", mod->ly_mod->name);
+            printf("==========================\n");
             if ((err_info = sr_shmsub_change_notify_evpipe(mod_info->conn->ext_shm.addr, mod, mod_info->ds,
                     SR_SUB_EV_DONE, cur_priority))) {
                 goto cleanup_wrunlock;

And the following debug changes to Netopeer2:

diff --git a/server/netconf_server.c b/server/netconf_server.c
index 64d200e8f667..641d34082cdc 100644
--- a/server/netconf_server.c
+++ b/server/netconf_server.c
@@ -188,6 +188,17 @@ np2srv_endpt_tcp_params_cb(sr_session_ctx_t *session, const char *UNUSED(module_
         /* find name */
         endpt_name = ((struct lyd_node_leaf_list *)node->parent->parent->parent->child)->value_str;
 
+#define CHANGE_NAME(c) ( \
+    (c == SR_OP_CREATED) ? "CREATED" : ( \
+        (c == SR_OP_MODIFIED) ? "DELETED" : ( \
+            (c == SR_OP_DELETED) ? "DELETED" : "???")))
+
+        char *dpath = lyd_path(node);
+        ERR("np2srv_endpt_tcp_params_cb %s: %s", CHANGE_NAME(op), dpath);
+        free(dpath);
+
+#undef CHANGE_NAME
+
         if (!strcmp(node->schema->name, "local-address")) {
             if ((op == SR_OP_CREATED) || (op == SR_OP_MODIFIED)) {
                 if (nc_server_endpt_set_address(endpt_name, ((struct lyd_node_leaf_list *)node)->value_str)) {
diff --git a/server/netconf_server_ssh.c b/server/netconf_server_ssh.c
index e61b3c254aca..2e9e27c02d72 100644
--- a/server/netconf_server_ssh.c
+++ b/server/netconf_server_ssh.c
@@ -281,15 +281,19 @@ np2srv_endpt_ssh_cb(sr_session_ctx_t *session, const char *UNUSED(module_name),
     while ((rc = sr_get_change_tree_next(session, iter, &op, &node, &prev_val, &prev_list, &prev_dflt)) == SR_ERR_OK) 
         /* get name */
         endpt_name = ((struct lyd_node_leaf_list *)node->parent->child)->value_str;
+        char *dpath = lyd_path(node);
 
         /* ignore other operations */
         if (op == SR_OP_CREATED) {
+            ERR("np2srv_endpt_ssh_cb CREATED: %s", dpath);
             rc = nc_server_add_endpt(endpt_name, NC_TI_LIBSSH);
             /* turn off all auth methods by default */
             nc_server_ssh_endpt_set_auth_methods(endpt_name, 0);
         } else if (op == SR_OP_DELETED) {
+            ERR("np2srv_endpt_ssh_cb DELETED: %s", dpath);
             rc = nc_server_del_endpt(endpt_name, NC_TI_LIBSSH);
         }
+        free(dpath);
         if (rc) {
             sr_free_change_iter(iter);
             return SR_ERR_INTERNAL;

When starting netopeer2-server the startup configuration is applied as expected:

user@tom:/tmp/netopeer$ sudo netopeer2-server -d &
[1] 2498
[ERR]: NP: np2srv_endpt_ssh_cb CREATED: /ietf-netconf-server:netconf-server/listen/endpoint[name='default-ssh']/ssh
[ERR]: NP: np2srv_endpt_tcp_params_cb CREATED: /ietf-netconf-server:netconf-server/listen/endpoint[name='default-ssh']/ssh/tcp-server-parameters/local-address
[ERR]: NP: np2srv_endpt_tcp_params_cb CREATED: /ietf-netconf-server:netconf-server/listen/endpoint[name='default-ssh']/ssh/tcp-server-parameters/keepalives
[ERR]: NP: np2srv_endpt_tcp_params_cb CREATED: /ietf-netconf-server:netconf-server/listen/endpoint[name='default-ssh']/ssh/tcp-server-parameters/local-port

And the server is listening as configured:

user@tom:/tmp/netopeer$ sudo ss -nlpt | grep netopeer
LISTEN   0   5    0.0.0.0:830      0.0.0.0:*      users:(("netopeer2-serve",pid=2499,fd=25))

However, when replacing the configuration by something different, the config change callbacks are not called (well, in some rare cases, they are called, but I did not manage to understand in which conditions).

user@tom:/tmp/netopeer$ sysrepocfg -m ietf-netconf-server -f xml -Xdefault.xml
user@tom:/tmp/netopeer$ cp default.xml two.xml
user@tom:/tmp/netopeer$ vim two.xml 
user@tom:/tmp/netopeer$ sysrepocfg -m ietf-netconf-server -f xml -Itwo.xml
sr_shmsub_change_notify_change_done: module=ietf-netconf-server
==========================
sr_shmsub_change_notify_change_done: module=ietf-netconf-server new configuration
<netconf-server xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-server">
  <listen>
    <endpoint>
      <name>ssh-default-port</name>
      <ssh>
        <tcp-server-parameters>
          <local-address>0.0.0.0</local-address>
          <keepalives>
            <idle-time>1</idle-time>
            <max-probes>10</max-probes>
            <probe-interval>5</probe-interval>
          </keepalives>
        </tcp-server-parameters>
        <ssh-server-parameters>
          <server-identity>
            <host-key>
              <name>default-key</name>
              <public-key>
                <keystore-reference>genkey</keystore-reference>
              </public-key>
            </host-key>
          </server-identity>
          <client-authentication>
            <supported-authentication-methods>
              <publickey/>
              <passsword/>
              <other>interactive</other>
            </supported-authentication-methods>
            <users/>
          </client-authentication>
        </ssh-server-parameters>
      </ssh>
    </endpoint>
    <endpoint>
      <name>ssh-ipv6-port-12345</name>
      <ssh>
        <tcp-server-parameters>
          <local-address>::</local-address>
          <local-port>12345</local-port>
          <keepalives>
            <idle-time>1</idle-time>
            <max-probes>10</max-probes>
            <probe-interval>5</probe-interval>
          </keepalives>
        </tcp-server-parameters>
        <ssh-server-parameters>
          <server-identity>
            <host-key>
              <name>default-key</name>
              <public-key>
                <keystore-reference>genkey</keystore-reference>
              </public-key>
            </host-key>
          </server-identity>
          <client-authentication>
            <supported-authentication-methods>
              <publickey/>
              <passsword/>
              <other>interactive</other>
            </supported-authentication-methods>
            <users/>
          </client-authentication>
        </ssh-server-parameters>
      </ssh>
    </endpoint>
  </listen>
</netconf-server>
sr_shmsub_change_notify_change_done: module=ietf-netconf-server notify subscribers
==========================
<<<<< NO LOG FROM NETOPEER2-SERVER >>>>
user@tom:/tmp/netopeer$ sudo ss -nlpt | grep netopeer
LISTEN   0   5    0.0.0.0:830      0.0.0.0:*      users:(("netopeer2-serve",pid=2499,fd=25))
<<<<< NO CHANGE IN SOCKETS >>>>>

Also, when applying an empty configuration:

user@tom:/tmp/netopeer$ vim empty.xml 
user@tom:/tmp/netopeer$ cat empty.xml 
<netconf-server xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-server">
</netconf-server>
user@tom:/tmp/netopeer$ sysrepocfg -m ietf-netconf-server -f xml -Iempty.xml
sr_shmsub_change_notify_change_done: module=ietf-netconf-server
==========================
sr_shmsub_change_notify_change_done: module=ietf-netconf-server new configuration
sr_shmsub_change_notify_change_done: module=ietf-netconf-server notify subscribers
==========================
<<<<< NO LOG FROM NETOPEER2-SERVER >>>>
user@tom:/tmp/netopeer$ sudo ss -nlpt | grep netopeer
LISTEN   0   5    0.0.0.0:830      0.0.0.0:*      users:(("netopeer2-serve",pid=2499,fd=25))
<<<<< NO CHANGE IN SOCKETS >>>>>

I don't know if I am doing something wrong (maybe the versions I am using are not well "aligned" with each other).

Thanks in advance for your help!

created time in 18 days

pull request commenthakimel/reveal.js

reveal.js 4.0.0

Hi @hakimel , pdf print seems broken on the dev branch.

All I have is a trace from chromium console:

print.js:44 Uncaught TypeError: this.Reveal.layoutSlideContents is not a function
    at value (print.js:44)
value @ print.js:44
load (async)
ve @ reveal.js:214
Promise.then (async)
ue @ reveal.js:135
Y.initialize @ index.js:38
(anonymous) @ ?print-pdf:31
hakimel

comment created time in 19 days

created tagrjarry/libyang-cffi

tagv1.0.166

Python CFFI bindings to libyang

created time in 20 days

push eventrjarry/libyang-cffi

Robin Jarry

commit sha 6733e6a693586d550eaf626d74d2d80e14cf44d2

mk: fix dev build in place Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 05ee0d8b09e1e13ba3c77257111ad6ee27a4fece

data: allow parsing rpc input Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha aeb155b882e033d87b5886f069f85073ed4d4644

data: fix parse_data_dict Behave differently if schema is a Module or a DNode. Add more unit tests. Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 4fa780af5288fad64bc354f292ea88ec819ba7cc

Update libyang C lib to version 1.0.166 Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

push time in 20 days

created tagrjarry/libyang-cffi

tagv1.0.165

Python CFFI bindings to libyang

created time in 21 days

push eventrjarry/libyang-cffi

Robin Jarry

commit sha 5efcfa7ca827474dd4adadc3752faa5a5a393ecc

Rename dump to print and str to mem The C functions use the "print" terminology. Use the same in the python binding instead of "dump". They also use the "mem" terminology. Use the same instead of "str". It makes more sense for the LYB format. Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 7c40e1b14e6496f170e7b3737fd521152c4c7ee8

data: handle dict format similar to others Consider the python dict format similar to the other serialization formats (JSON, XML, LYB). Add print_dict() to DNode (code is the same than dnode_to_dict). Add parse_data_dict() to Module and SNode (calls dict_to_dnode). The remaining dict_to_dnode is now "internal" API. Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 684a96c1eb1e030c551a8703fbe514b98841762b

Update libyang C lib to version 1.0.165 Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

push time in 21 days

issue closedsysrepo/sysrepo

sr_get_data returns libyang error during SR_EV_CHANGE with leaf-list default values

Hi Michal,

sorry about the long title, I tried to make it as concise as possible. More complete description of the problem follows.

Given the following yang model:

module example {
  yang-version 1.1;
  namespace "urn:example";
  prefix example;

  typedef foo {
    type enumeration {
      enum "bar1";
      enum "bar2";
      enum "bar3";
    }
  }

  container config {
    container system {
      presence "";
      leaf enabled {
        type boolean;
        default true;
      }
      leaf-list val {
        type foo;
        default "bar1";
        default "bar2";
        default "bar3";
      }
    }
  }
}

And the following patch applied to examples/application_changes_example.c:

diff --git a/examples/application_changes_example.c b/examples/application_changes_example.c
index 9f561f465753..1eea523c14d5 100644
--- a/examples/application_changes_example.c
+++ b/examples/application_changes_example.c
@@ -137,26 +137,23 @@ print_change(sr_change_oper_t op, sr_val_t *old_val, sr_val_t *new_val)
     }
 }
 
-static void
-print_current_config(sr_session_ctx_t *session, const char *module_name)
+static int
+print_current_config(sr_session_ctx_t *session, const char *xpath)
 {
-    sr_val_t *values = NULL;
-    size_t count = 0;
-    int rc = SR_ERR_OK;
-    char *xpath;
+    struct lyd_node *config = NULL;
+    int rc;
 
-    asprintf(&xpath, "/%s:*//.", module_name);
-
-    rc = sr_get_items(session, xpath, 0, 0, &values, &count);
-    free(xpath);
-    if (rc != SR_ERR_OK) {
-        return;
+    if ((rc = sr_get_data(session, xpath, 0, 0, 0, &config)) != SR_ERR_OK) {
+        printf("failed to get data: %s\n", sr_strerror(rc));
+        return rc;
     }
-
-    for (size_t i = 0; i < count; i++){
-        print_val(&values[i]);
+    if (lyd_print_file(stdout, config, LYD_XML, LYP_FORMAT | LYP_WD_ALL)) {
+        printf("failed to print config\n");
+        return SR_ERR_LY;
     }
-    sr_free_values(values, count);
+    lyd_free_withsiblings(config);
+
+    return SR_ERR_OK;
 }
 
 const char *
@@ -202,10 +199,10 @@ module_change_cb(sr_session_ctx_t *session, const char *module_name, const char
 
     printf("\n ========== END OF CHANGES =======================================");
 
-    if (event == SR_EV_DONE) {
-        printf("\n\n ========== CONFIG HAS CHANGED, CURRENT RUNNING CONFIG: ==========\n\n");
-        print_current_config(session, module_name);
-    }
+        printf("\n\n ========== CONFIG %s: ==========\n\n", ev_to_str(event));
+        if ((rc = print_current_config(session, xpath)) != SR_ERR_OK) {
+            return rc;
+        }
 
 cleanup:
     sr_free_change_iter(it);
@@ -257,7 +254,7 @@ main(int argc, char **argv)
 
     /* read current config */
     printf("\n ========== READING RUNNING CONFIG: ==========\n\n");
-    print_current_config(session, mod_name);
+    print_current_config(session, xpath);
 
     /* subscribe for changes in running config */
     rc = sr_module_change_subscribe(session, mod_name, xpath, module_change_cb, NULL, 0, 0, &subscription);

After installing the yang model, the application is started:

~# sysrepoctl -a -i example.yang
~# ./examples/application_changes_example example /example:config &
[1] 5736
Application will watch for changes in "/example:config".

 ========== READING RUNNING CONFIG: ==========

~# cat > config.xml
<config xmlns="urn:example">
  <system>
    <enabled>true</enabled>
    <val>bar1</val>
  </system>
</config>
~# sysrepocfg --import=config.xml -m example -d running

 ========== EVENT change CHANGES: ====================================

CREATED: /example:config/system (container)
CREATED: /example:config/system/enabled = true
CREATED: /example:config/system/val = bar1

 ========== END OF CHANGES =======================================

 ========== CONFIG change: ==========

<config xmlns="urn:example">
  <system>
    <enabled>true</enabled>
    <val>bar1</val>
  </system>
</config>


 ========== EVENT done CHANGES: ====================================

CREATED: /example:config/system (container)
CREATED: /example:config/system/enabled = true
CREATED: /example:config/system/val = bar1

 ========== END OF CHANGES =======================================

 ========== CONFIG done: ==========

<config xmlns="urn:example">
  <system>
    <enabled>true</enabled>
    <val>bar1</val>
  </system>
</config>

~# cat > config.xml
<config xmlns="urn:example">
  <system>
    <enabled>false</enabled>
  </system>
</config>
~# sysrepocfg --import=config.xml -m example -d running

 ========== EVENT change CHANGES: ====================================

MODIFIED: /example:config/system/enabled = true
to /example:config/system/enabled = false
CREATED: /example:config/system/val = bar2 [default]
CREATED: /example:config/system/val = bar3 [default]

 ========== END OF CHANGES =======================================

 ========== CONFIG change: ==========

[ERR]: Insert request refers node (/example:config/system/val[.='bar1']) that is going to be auto-deleted.
failed to get data: libyang error
[ERR]: libyang error
sysrepocfg error: Replace config failed (User callback failed)

If sr_get_data is called only on SR_EV_DONE, there is no error. The problem occurs only when calling sr_get_data on SR_EV_CHANGE when there are leaf-list with default values that are "restored".

I ran the example with gdb to spot the exact cause for the problem, it occurs here:

https://github.com/sysrepo/sysrepo/blob/master/src/edit_diff.c#L2571

Thanks in advance for your help.

closed time in 25 days

rjarry

issue commentsysrepo/sysrepo

sr_get_data returns libyang error during SR_EV_CHANGE with leaf-list default values

Thanks Michal, it works now!

rjarry

comment created time in 25 days

issue commentsysrepo/sysrepo

sr_get_data returns libyang error during SR_EV_CHANGE with leaf-list default values

Same problem occurs with the following versions:

https://github.com/CESNET/libyang/commit/1c0e2761fcbabd7a346ea7b42e064f6fe0463752 https://github.com/sysrepo/sysrepo/commit/7784c41fc0e180a329922efd4edd3d01eaa4f1a8

rjarry

comment created time in a month

issue commentsysrepo/sysrepo

sr_get_data returns libyang error during SR_EV_CHANGE with leaf-list default values

Hi Michal,

I am having the problem with version: 5d5ac78e1247 (devel branch, from April 16th). I am trying with the latest version.

rjarry

comment created time in a month

created tagrjarry/libyang-cffi

tagv1.0.164

Python CFFI bindings to libyang

created time in a month

delete branch rjarry/libyang-cffi

delete branch : nostatic

delete time in a month

delete branch rjarry/libyang-cffi

delete branch : candidate

delete time in a month

push eventrjarry/libyang-cffi

Robin Jarry

commit sha b527e8fe4b80f12ee34b6145fb63339dd4100d40

log: disable logging messages by default This may disturb other applications running with libyang. By default, only store the error messages and use the first stored error when raising a context error. Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 474d53432528a9575f25811dc350ca131a026627

dict_to_dnode: add checks for python values Instead of obscure AttributeError or KeyErrors, raise explicit proper errors with explicit messages when the provided python values do not match the expected ones in the schema. Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 55669e1e2ef22bb2862bf4f61e8f7412975ebde9

data: remove unsupported methods The dflt struct field is not available in the mapped structure. Bit fields are not handled properly by cffi for incomplete structures (i.e. with '...'). Remove DNode.default(). Also, lyd_change_leaf has not been exported in the CFFI binding, remove DLeaf.set_value(). Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha cc02e296f751181e09df19ecc071370c9231b019

Update libyang C lib to version 1.0.164 Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha d31ff4856b1cc3b193440d7a2ccb41376051172b

context: add parse_module_* methods Add functions to parse a YANG module from a file or from a string. Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha cd1210e448d8ab303126fda06204763a90e1d3ac

data: remove DNode autofree parameter This parameter adds the risk of double free. If the user wants to free a data node, they should do so explicitly. Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha ac2d7a994e9c50cc5ba61f63dc1e5fd430385332

tests: make sure to free allocated data nodes Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha d09f1c27562322933a1de7ec572485d07feaf401

all: use standard python arguments instead of C flags Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 524af21dcbd0bc392dc0e7efb2fe7e61af4f668e

data: add root method Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha f17f65398d961f2e6bbcb672d73c6437738b1ce2

data: add *siblings methods Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 65cf697e199bd98a27ac7246c6ccd340030430d9

dict_to_dnode: always create containers Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 918ca102dbae31a5a28d46aa6ecbd487a722e041

errors: remove duplicate path Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha a8408d3d85a7d143a5266ad1a7e62f7333b2aad1

data: do not try to convert lyb format to unicode Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 6f4811eefde1d87a4a75b968b639ab55f1dc1cf9

data: always return something in DLeaf.value Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 2432da82908ae7a1fb343302500af8e447e8baa9

dnode_to_dict: support standard printer options Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 6dae8ee38fb3c80149a2a9a1ba07c50494dcd43c

context: allow explicitly destroying Instead of automatically calling ly_ctx_destroy when the last reference to self._ctx is lost, add a destroy() method. This method can be called multiple times without risking a "double free". Add context manager methods so that Context() can be used in with constructs: with Context() as ctx: # do stuff with ctx # ctx has been destroyed After destroy() has been called the context may not be used anymore. Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 3966e55f1f04b138de7fdced1b84679b598d927f

schema: add module.implemented() lys_node->implemented is a bit field. CFFI does not handle bit fields included in partial structure maps (i.e. with '...'). Add a trivial C function to access to the structure field. Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

Robin Jarry

commit sha 47a897f5d5e871c1c3651ba751d37f49662680d9

setup: do not embed libyang.so by default Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

push time in a month

create barnchrjarry/libyang-cffi

branch : candidate

created branch time in a month

issue openedsysrepo/sysrepo

sr_get_data returns libyang error during SR_EV_CHANGE with leaf-list default values

Hi Michal,

sorry about the long title, I tried to make it as concise as possible. More complete description of the problem follows.

Given the following yang model:

module example {
  yang-version 1.1;
  namespace "urn:example";
  prefix example;

  typedef foo {
    type enumeration {
      enum "bar1";
      enum "bar2";
      enum "bar3";
    }
  }

  container config {
    container system {
      presence "";
      leaf enabled {
        type boolean;
        default true;
      }
      leaf-list val {
        type foo;
        default "bar1";
        default "bar2";
        default "bar3";
      }
    }
  }
}

And the following patch applied to examples/application_changes_example.c:

diff --git a/examples/application_changes_example.c b/examples/application_changes_example.c
index 9f561f465753..1eea523c14d5 100644
--- a/examples/application_changes_example.c
+++ b/examples/application_changes_example.c
@@ -137,26 +137,23 @@ print_change(sr_change_oper_t op, sr_val_t *old_val, sr_val_t *new_val)
     }
 }
 
-static void
-print_current_config(sr_session_ctx_t *session, const char *module_name)
+static int
+print_current_config(sr_session_ctx_t *session, const char *xpath)
 {
-    sr_val_t *values = NULL;
-    size_t count = 0;
-    int rc = SR_ERR_OK;
-    char *xpath;
+    struct lyd_node *config = NULL;
+    int rc;
 
-    asprintf(&xpath, "/%s:*//.", module_name);
-
-    rc = sr_get_items(session, xpath, 0, 0, &values, &count);
-    free(xpath);
-    if (rc != SR_ERR_OK) {
-        return;
+    if ((rc = sr_get_data(session, xpath, 0, 0, 0, &config)) != SR_ERR_OK) {
+        printf("failed to get data: %s\n", sr_strerror(rc));
+        return rc;
     }
-
-    for (size_t i = 0; i < count; i++){
-        print_val(&values[i]);
+    if (lyd_print_file(stdout, config, LYD_XML, LYP_FORMAT | LYP_WD_ALL)) {
+        printf("failed to print config\n");
+        return SR_ERR_LY;
     }
-    sr_free_values(values, count);
+    lyd_free_withsiblings(config);
+
+    return SR_ERR_OK;
 }
 
 const char *
@@ -202,10 +199,10 @@ module_change_cb(sr_session_ctx_t *session, const char *module_name, const char
 
     printf("\n ========== END OF CHANGES =======================================");
 
-    if (event == SR_EV_DONE) {
-        printf("\n\n ========== CONFIG HAS CHANGED, CURRENT RUNNING CONFIG: ==========\n\n");
-        print_current_config(session, module_name);
-    }
+        printf("\n\n ========== CONFIG %s: ==========\n\n", ev_to_str(event));
+        if ((rc = print_current_config(session, xpath)) != SR_ERR_OK) {
+            return rc;
+        }
 
 cleanup:
     sr_free_change_iter(it);
@@ -257,7 +254,7 @@ main(int argc, char **argv)
 
     /* read current config */
     printf("\n ========== READING RUNNING CONFIG: ==========\n\n");
-    print_current_config(session, mod_name);
+    print_current_config(session, xpath);
 
     /* subscribe for changes in running config */
     rc = sr_module_change_subscribe(session, mod_name, xpath, module_change_cb, NULL, 0, 0, &subscription);

After installing the yang model, the application is started:

~# sysrepoctl -a -i example.yang
~# ./examples/application_changes_example example /example:config &
[1] 5736
Application will watch for changes in "/example:config".

 ========== READING RUNNING CONFIG: ==========

~# cat > config.xml
<config xmlns="urn:example">
  <system>
    <enabled>true</enabled>
    <val>bar1</val>
  </system>
</config>
~# sysrepocfg --import=config.xml -m example -d running

 ========== EVENT change CHANGES: ====================================

CREATED: /example:config/system (container)
CREATED: /example:config/system/enabled = true
CREATED: /example:config/system/val = bar1

 ========== END OF CHANGES =======================================

 ========== CONFIG change: ==========

<config xmlns="urn:example">
  <system>
    <enabled>true</enabled>
    <val>bar1</val>
  </system>
</config>


 ========== EVENT done CHANGES: ====================================

CREATED: /example:config/system (container)
CREATED: /example:config/system/enabled = true
CREATED: /example:config/system/val = bar1

 ========== END OF CHANGES =======================================

 ========== CONFIG done: ==========

<config xmlns="urn:example">
  <system>
    <enabled>true</enabled>
    <val>bar1</val>
  </system>
</config>

~# cat > config.xml
<config xmlns="urn:example">
  <system>
    <enabled>false</enabled>
  </system>
</config>
~# sysrepocfg --import=config.xml -m example -d running

 ========== EVENT change CHANGES: ====================================

MODIFIED: /example:config/system/enabled = true
to /example:config/system/enabled = false
CREATED: /example:config/system/val = bar2 [default]
CREATED: /example:config/system/val = bar3 [default]

 ========== END OF CHANGES =======================================

 ========== CONFIG change: ==========

[ERR]: Insert request refers node (/example:config/system/val[.='bar1']) that is going to be auto-deleted.
failed to get data: libyang error
[ERR]: libyang error
sysrepocfg error: Replace config failed (User callback failed)

If sr_get_data is called only on SR_EV_DONE, there is no error. The problem occurs only when calling sr_get_data on SR_EV_CHANGE when there are leaf-list with default values that are "restored".

I ran the example with gdb to spot the exact cause for the problem, it occurs here:

https://github.com/sysrepo/sysrepo/blob/master/src/edit_diff.c#L2571

Thanks in advance for your help.

created time in a month

PR opened CESNET/Netopeer2

server BUGFIX fix partial processing of tcp params changes

When deleting an enpoint, neither nc_server_endpt_set_keepalives nor nc_server_ch_client_endpt_set_keepalives are called. rc remains with its current value which is necessarily SR_ERR_NOT_FOUND (meaning "end of changes").

This causes np2srv_tcp_keepalives to return SR_ERR_INTERNAL which causes np2srv_endpt_tcp_params_cb to stop processing the changes and return an error. No error message is printed.

Reset the value of rc to fix the issue.

If one of nc_server_ch_client_endpt_set_keepalives or nc_server_endpt_set_keepalives actually failed, display an explicit error message.

+3 -0

0 comment

1 changed file

pr created time in a month

create barnch6WIND/Netopeer2

branch : tcp-params-keepalives

created branch time in a month

issue commentsysrepo/sysrepo

Proposal: "more pythonic" bindings

@rkrejci,

The bindings I am working on are still under development but here is a bit of dummy example code:

# server
import asyncio
import signal
import libyang
import libsysrepo

async def module_change_cb(xpath, event, config, changes, private_data):
    if event != libsysrepo.Event.DONE:
        return
    hostname = config['config']['hostname']
    proc = await asyncio.create_subprocess_exec(
        ['hostnamectl', 'set-hostname', hostname])
    await proc.wait()

stop_event = asyncio.Event()
loop = asyncio.get_event_loop()
loop.add_signal_handler(signal.SIGTERM, stop_event.set)
loop.add_signal_handler(signal.SIGINT, stop_event.set)
loop.add_signal_handler(signal.SIGQUIT, stop_event.set)

with libsysrepo.Connection() as conn:
    with conn.start_session() as sess:
        sess.subscribe_module_change(
            'mymodule', '/mymodule:config/hostname', module_change_cb)
        loop.run_until_complete(stop_event.wait())
# client
>>> import libyang
>>> import libsysrepo
>>> with libsysrepo.Connection() as conn:
>>>     with conn.start_session(libsysrepo.Datastore.RUNNING) as sess:
>>>         config = sess.get_subtree_dict('/mymodule:config')
>>>         print('==== current config ====')
>>>         print(config)
>>>         config['config']['hostname'] = 'foobar'
>>>         dnode = libyang.dict_to_dnode(config, conn.get_ly_ctx().get_module('mymodule'))
>>>         sess.replace_config(dnode)
>>>         config = sess.get_subtree_dict('/mymodule:config')
>>>         print('==== updated config ====')
>>>         print(config)
...
==== current config ====
{'config': {'hostname': 'localhost'}}
==== updated config ===='
{'config': {'hostname': 'foobar'}}
sinill57

comment created time in a month

issue commentsysrepo/sysrepo

Proposal: "more pythonic" bindings

I forgot to mention. One upside in using CFFI instead of manually written python C extensions is pypy support directly out of the box.

sinill57

comment created time in a month

issue commentsysrepo/sysrepo

Proposal: "more pythonic" bindings

Hi Radek,

I have managed to do that. The libyang objects returned by libsysrepo.so, can be fed directly to the CFFI binding I made for libyang.so. A simple cast is required: https://github.com/rjarry/libyang-cffi/blob/master/libyang/data.py#L164

Also, libyang objects generated from python, can be fed back to libsysrepo.so (casting the other way around).

If you are interested in all this, I can start cleaning the libsysrepo CFFI bindings I am working on and show you what it looks like.

In any case, if we are talking about integrating this into libyang/sysrepo, it would require some work. More work if you want to get rid of CFFI since it takes care of most of the boilerplate needed to map C objects to python objects (and back).

Please let me know what you think.

Cheers,

sinill57

comment created time in a month

issue commentsysrepo/sysrepo

Proposal: "more pythonic" bindings

Hi Michal,

I only used CFFI as a convenience. The same result should be achievable without it, albeit with more work.

In any case, if the "pythonic" API seems OK to you, I could have a look to get rid of CFFI.

sinill57

comment created time in a month

issue commentsysrepo/sysrepo

Proposal: "more pythonic" bindings

@rkrejci I have written CFFI bindings for libyang which we use extensively for our projects. SWIG had too much drawbacks. CFFI allows better integration with the python ecosystem (virtualenv, pip, etc.).

Following this, I am integrating the new sysrepo API and I have written a CFFI binding for it (again no SWIG).

These bindings are incomplete and under development but I would be glad to share them (and maybe integrate them into sysrepo and libyang if possible).

Note : the sysrepo bindings are asyncio-compatible.

sinill57

comment created time in a month

issue commentsysrepo/sysrepo

Modules management improvements

Hi Michal,

Based on your changes it seems to me you mostly want to ignore all the errors. That can be achieved by simply ignoring the return value of sysrepoctl and turn its logging off (-v0) so I think those changes are pointless.

I am not ignoring all errors. Only ignoring certain ones. Ignoring all errors would not be desirable. I want to be able to see if the module cannot be installed because of some real problem (missing dependency, parsing error, etc.). I only want to ignore the module already installed, revision is the same or module already removed errors (I do not consider these exactly "errors").

Ideally, these operations should be idempotent (i.e.: running sr_module_install_force() twice with the same module should not produce any warning). My patch achieves exactly that.

I am willing to rework it if necessary. In any case, we need something of the sort. I'll keep it in our local changes in the meantime.

Thanks for taking the time to look at it.

rjarry

comment created time in a month

issue closedsysrepo/sysrepo

Change the format of data files in /etc/sysrepo/data

Hello,

The cmake FILE_FORMAT_EXT build option is not available anymore. Could it be possible to add a build option to choose the format of the data files in the master and devel branches?

Thanks in advance.

closed time in a month

rjarry

issue commentsysrepo/sysrepo

Change the format of data files in /etc/sysrepo/data

I understand. We'll try to manage with yanglint.

rjarry

comment created time in a month

issue openedsysrepo/sysrepo

Change the format of data files in /etc/sysrepo/data

Hello,

The cmake FILE_FORMAT_EXT build option is not available anymore. Could it be possible to add a build option to choose the format of the data files in the master and devel branches?

Thanks in advance.

created time in a month

issue commentsysrepo/sysrepo

Modules management improvements

I have made a draft patch to address the issues I mentioned here. Could you please tell me what you think about it. And maybe merge the pull request if that looks ok to you :)

Thanks in advance.

rjarry

comment created time in a month

PR opened sysrepo/sysrepo

Add force functions for modules management

Add new functions for YANG modules management.

sr_install_module_force

Does the same than sr_install_module but will ignore if the module is already installed (no error message will be logged). If a different version of the module is already installed, it will be updated.

sr_remove_module_force

Does the same than sr_remove_module but will ignore if the module is not installed or already scheduled for deletion (no error message will be logged).

sr_update_module_force

Does the same than sr_update_module but will ignore if the new module has the same revision than before. It will be updated anyway.

All these functions are only useful in development.

Add -f/--force option to sysrepoctl that uses these functions.

Fixes: #1917

+170 -28

0 comment

5 changed files

pr created time in a month

create barnch6WIND/sysrepo

branch : modules-management

created branch time in a month

PR opened sysrepo/sysrepo

Disable cwd searchdir in libyang context

YANG models should only be searched in sysrepo YANG dir.

+1 -1

0 comment

1 changed file

pr created time in a month

create barnch6WIND/sysrepo

branch : ly-ctx-no-cwd

created branch time in a month

PR opened sysrepo/sysrepo

Add editorconfig file

EditorConfig is a file format and collection of text editor plugins for maintaining consistent coding styles between different editors and IDEs.

Initialize the file following the current coding style in existing files.

In order for this file to be taken into account (unless they use an editor with built-in EditorConfig support), developers will have to install a plugin.

See the following links for more details.

https://editorconfig.org/ https://github.com/editorconfig/editorconfig-emacs https://github.com/editorconfig/editorconfig-vim

+18 -0

0 comment

1 changed file

pr created time in a month

create barnch6WIND/sysrepo

branch : editorconfig

created branch time in a month

issue closedCESNET/libnetconf2

yang 1.1 modules not returned in server capabilities

After looking at the code:

https://github.com/CESNET/libnetconf2/blob/devel/src/session.c#L1109

YANG modules with yang-version 1.1; are not returned in the server advertised capabilities. Is there a way to return all implemented modules in the libyang context?

closed time in a month

rjarry

issue commentCESNET/libnetconf2

yang 1.1 modules not returned in server capabilities

With the latest version on the devel branch, it seems fixed. I may have had a broken environment for a while. Sorry about the disturbance :)

rjarry

comment created time in a month

issue commentCESNET/libnetconf2

yang 1.1 modules not returned in server capabilities

It seems like these 1.1 modules are not in the libyang context returned by nc_session_get_ctx().

rjarry

comment created time in a month

issue commentsysrepo/sysrepo

Modules management improvements

Hi Michal,

I have absolutely no issue if reworking all our scripts from scratch if that is required. My main goal is to make it work. Whatever the cost :)

Let me clarify our use case: we have approximately 70 YANG modules. For modularity purposes, we have a base module which contains almost nothing but two root containers. All other modules augment the base module with additional data and/or rpcs. This means a lot of dependencies between these modules.

When working in a development environment, every time we make a slight modification in one of the modules, we need to re-install(update) the module in sysrepo.

Until now (with the old sysrepo), we managed that with sysrepoctl commands. Since module update was flaky, we needed to completely uninstall all our modules first, and then reinstall them all. Only uninstalling our base module was not possible because of dependencies.

Now that there is an --update option, I tried to use it but it would mean adding a new (dummy) revision to modules for each little change made during development. I understand the need for revisions to identify changes in modules, but this has been designed with "production" environment in mind. This is not practical when in a development environment where we make (temporary and minor) modifications to YANG modules multiple times per hour.

In any case, I gave up with this --update option and tried to uninstall/install all modules, like it was done before. However, it does not work either. The uninstallation fails because of dependency problems. It seems I am stuck :)

To sum-up our need: We need to be able to do micro updates of our modules during the development phase without requiring a new YANG revision every time. If that means uninstalling everything every time, we can live with that.

I am willing to share our complete set of YANG modules and more details about the exact errors if you want.

Thanks in advance for your help.

As a side note:

sysrepoctl --install for an already installed module (in the same revision) prints an error but should actually return 0,

This is very misleading. If there is no problem, the error should not be printed. I see that when running make install multiple times in Netopeer2:

~# make install
...
-- Installing sysrepo modules...
[ERR]: Module "ietf-netconf-acm" not scheduled for deletion.
[ERR]: Module "ietf-netconf-acm" is already in sysrepo.
[ERR]: Module "ietf-netconf@2013-09-29" already installed.
sysrepoctl error: Failed to update module "/root/sshfs_root/Netopeer2/server/../modules/ietf-netconf@2013-09-29.yang" (Item already exists)
-- Generating a new RSA host key "genkey"...
-- Merging default server listen configuration...
rjarry

comment created time in a month

issue closedsysrepo/sysrepo

Segmentation fault when a rpc callback returns SR_ERR_CALLBACK_SHELVE

When an RPC callback returns SR_ERR_CALLBACK_SHELVE and sr_process_events is called a second time to finalize the operation (and return the result), a segmentation fault occurs when trying to access a libyang parsing error.

Program terminated with signal SIGSEGV, Segmentation fault.
#0  sr_errinfo_new_ly (err_info=err_info@entry=0x7fffe28ae400, ly_ctx=0x28da9c0)
    at /root/sshfs_root/sysrepo/src/log.c:217
217             if (e->level == LY_LLWRN) {
#1  0x00007f20d8bc91fe in sr_shmsub_rpc_listen_process_rpc_events (rpc_subs=0x3852f30, conn=0x2982f60)
    at /root/sshfs_root/sysrepo/src/shm_sub.c:2513
#2  0x00007f20d8ba1ea2 in sr_process_events (subscription=0x381bad0, session=0x0, stop_time_in=0x0)
    at /root/sshfs_root/sysrepo/src/sysrepo.c:2978
...

I think the segfault occurs because I have registered a log callback for libyang which causes ly_err_first to return NULL because the error has already been consumed.

In any case, this segfault hides another problem. The RPC input fails to be parsed a second time when sr_process_events is called again when the (async) RPC callback has finished its work and tries to return the result.

I don't know how to fix this. Could you help?

Thanks in advance.

closed time in a month

rjarry

issue closedsysrepo/sysrepo

Cannot subscribe async callbacks with SR_SUBSCR_ENABLED

Hi Michal,

When using our own event loop (i.e. callbacks are scheduled for later, and return SR_ERR_CALLBACK_SHELVE when they are not finished), there is no way to subscribe to module changes with the SR_SUBSCR_ENABLED option. Here is a dummy example:

>>> sr_connect()
Applying scheduled changes.
No scheduled changes.
>>> sr_session_start()
Session 30 (user "root") created.
>>> sr_module_change_subscribe("example", my_async_callback, SR_SUBSCR_ENABLED)
Triggering "example" "enabled" event on enabled data.
>>> my_async_callback called with SR_EV_ENABLED, returns SR_ERR_CALLBACK_SHELVE
Subscribing to "example" changes failed.

Looking at the code, it seems like the SR_EV_DONE callback is also not expected to return SR_ERR_CALLBACK_SHELVE (even though the return code is ignored).

https://github.com/sysrepo/sysrepo/blob/master/src/sysrepo.c#L3160-L3178

I don't know what could be done to fix that problem. Ideally, these callbacks should be allowed to return SR_ERR_CALLBACK_SHELVE and the application running its own event loop should be allowed to call sr_process_events when the callbacks have finished running.

I understand this is complex as the SR_EV_ENABLED and SR_EV_DONE events are simulated upon the call of sr_module_change_subscribe(). If the callback returns SR_ERR_CALLBACK_SHELVE, there should be some kind of flag telling the subscription is still "pending" until the callback has called sr_process_events for both SR_EV_ENABLED and SR_EV_DONE.

Thanks in advance for your help.

closed time in a month

rjarry

issue commentsysrepo/sysrepo

Cannot subscribe async callbacks with SR_SUBSCR_ENABLED

Thanks Michal.

rjarry

comment created time in a month

issue openedsysrepo/sysrepo

Modules management improvements

Hi Michal,

In the legacy branch, sysrepoctl was able to re-install an already installed module. This is not possible anymore. sysrepoctl --install only works if the module is not installed. If the module is already installed with the same revision (or even with a different revision), it fails.

And sysrepoctl --update does not work when the revisions did not change. This is quite tedious when working as we need to add a revision every time we change small things.

Would you consider adding a --force option to both sysrepoctl --install and sysrepoctl --update (and also maybe sysrepoctl --remove) ?

Ideally, there should be a way to do sysrepoctl --force --install-or-update my-module.yang which would do what's required. This would avoid overly complex logic in shell scripts. I figure the --force and the --install-or-update options would be reflected in the C API to use without sysrepoctl.

If that seems ok for you, I can try to do a draft patch.

Thanks in advance.

created time in a month

issue commentsysrepo/sysrepo

Segmentation fault when a rpc callback returns SR_ERR_CALLBACK_SHELVE

For the record, the last error message that was printed by libyang before the segfault is:

Invalid arguments (lyd_schema_sort()).

Which occurs when trying to parse the RCP input parameters for the second time:

https://github.com/sysrepo/sysrepo/blob/master/src/shm_sub.c#L2510

rjarry

comment created time in a month

issue openedCESNET/libnetconf2

yang 1.1 modules not returned in server capabilities

After looking at the code:

https://github.com/CESNET/libnetconf2/blob/devel/src/session.c#L1109

YANG modules with yang-version 1.1; are not returned in the server advertised capabilities. Is there a way to return all implemented modules in the libyang context?

created time in a month

issue commentsysrepo/sysrepo

Segmentation fault when a rpc callback returns SR_ERR_CALLBACK_SHELVE

This (naive) patch fixes the problem:

diff --git a/src/log.c b/src/log.c
index 1be87ce08746..1485e8a4a518 100644
--- a/src/log.c
+++ b/src/log.c
@@ -210,8 +210,8 @@ sr_errinfo_new_ly(sr_error_info_t **err_info, struct ly_ctx *ly_ctx)
     struct ly_err_item *e;
 
     e = ly_err_first(ly_ctx);
-    /* this function is called only when an error is expected */
-    assert(e);
+    if (!e)
+           return;
 
     do {
         if (e->level == LY_LLWRN) {
diff --git a/src/shm_sub.c b/src/shm_sub.c
index 4af86bb2f177..3302ae189bc5 100644
--- a/src/shm_sub.c
+++ b/src/shm_sub.c
@@ -2578,7 +2578,7 @@ process_event:
                 /* this subscription did not process the event yet, skip it */
                 SR_LOG_INF("Shelved processing \"%s\" event with ID %u priority %u.",
                         sr_ev2str(multi_sub_shm->event), multi_sub_shm->request_id, multi_sub_shm->priority);
-                continue;
+                goto cleanup_rdunlock;
             } else if (timed_out || (ret != SR_ERR_OK)) {
                 /* whole event failed */
                 err_code = ret;
rjarry

comment created time in a month

issue commentsysrepo/sysrepo

Cannot subscribe async callbacks with SR_SUBSCR_ENABLED

I understand the complexity behind all this.

I think the best workaround (for now) is to return immediately when called with SR_EV_ENABLED and not defer the callback in that case. Maybe limitations around SR_ERR_CALLBACK_SHELVE should be mentioned in the docs.

Thanks for the reply.

rjarry

comment created time in a month

issue openedsysrepo/sysrepo

Segmentation fault when a rpc callback returns SR_ERR_CALLBACK_SHELVE

When an RPC callback returns SR_ERR_CALLBACK_SHELVE and sr_process_events is called a second time to finalize the operation (and return the result), a segmentation fault occurs when trying to access a libyang parsing error.

Program terminated with signal SIGSEGV, Segmentation fault.
#0  sr_errinfo_new_ly (err_info=err_info@entry=0x7fffe28ae400, ly_ctx=0x28da9c0)
    at /root/sshfs_root/sysrepo/src/log.c:217
217             if (e->level == LY_LLWRN) {
#1  0x00007f20d8bc91fe in sr_shmsub_rpc_listen_process_rpc_events (rpc_subs=0x3852f30, conn=0x2982f60)
    at /root/sshfs_root/sysrepo/src/shm_sub.c:2513
#2  0x00007f20d8ba1ea2 in sr_process_events (subscription=0x381bad0, session=0x0, stop_time_in=0x0)
    at /root/sshfs_root/sysrepo/src/sysrepo.c:2978
...

I think the segfault occurs because I have registered a log callback for libyang which causes ly_err_first to return NULL because the error has already been consumed.

In any case, this segfault hides another problem. The RPC input fails to be parsed a second time when sr_process_events is called again when the (async) RPC callback has finished its work and tries to return the result.

I don't know how to fix this. Could you help?

Thanks in advance.

created time in a month

issue openedsysrepo/sysrepo

Cannot subscribe async callbacks with SR_SUBSCR_ENABLED

Hi Michal,

When using our own event loop (i.e. callbacks are scheduled for later, and return SR_ERR_CALLBACK_SHELVE when they are not finished), there is no way to subscribe to module changes with the SR_SUBSCR_ENABLED option. Here is a dummy example:

>>> sr_connect()
Applying scheduled changes.
No scheduled changes.
>>> sr_session_start()
Session 30 (user "root") created.
>>> sr_module_change_subscribe("example", my_async_callback, SR_SUBSCR_ENABLED)
Triggering "example" "enabled" event on enabled data.
>>> my_async_callback called with SR_EV_ENABLED, returns SR_ERR_CALLBACK_SHELVE
Subscribing to "example" changes failed.

Looking at the code, it seems like the SR_EV_DONE callback is also not expected to return SR_ERR_CALLBACK_SHELVE (even though the return code is ignored).

https://github.com/sysrepo/sysrepo/blob/master/src/sysrepo.c#L3163

I don't know what could be done to fix that problem. Ideally, these callbacks should be allowed to return SR_ERR_CALLBACK_SHELVE and the application running its own event loop should be allowed to call sr_process_events when the callbacks have finished running.

I understand this is complex as the SR_EV_ENABLED and SR_EV_DONE events are simulated upon the call of sr_module_change_subscribe(). Ideally, if the callback returns SR_ERR_CALLBACK_SHELVE, there should be some kind of flag telling the subscription is still "pending" until the callback has called sr_process_events for both SR_EV_ENABLED and SR_EV_DONE.

Thanks in advance for your help.

created time in a month

issue closedsysrepo/sysrepo

Cannot use implicit session after returning SR_ERR_CALLBACK_SHELVE

Hi Michal,

we finally have time to start integrating the new sysrepo API. Doing so, I encountered an issue with using SR_SUBSCR_NO_THREAD along with our own event loop.

In a nutshell, when using an event loop, potentially blocking operations are deferred to allow other tasks to run. However, the "implicit session" provided as argument in the module change callback cannot be used outside of the callback scope which prevents from scheduling the processing of changes to later.

Here is a patch on examples/application_changes_example.c to illustrate the problem. I used libevent to schedule sr_process_events when the subscription event pipe becomes readable. In the module change callback, I simulate an operation that takes 1 second with a timer event which schedules another callback in the event loop with a copy of the arguments (including the "implicit session").

diff --git a/examples/CMakeLists.txt b/examples/CMakeLists.txt
index cde805f55ba4..937858055cd1 100644
--- a/examples/CMakeLists.txt
+++ b/examples/CMakeLists.txt
@@ -9,7 +9,7 @@ set(examples application_changes_example sr_set_item_example sr_get_items_exampl
 
 foreach(app_name IN LISTS examples)
     add_executable(${app_name} ${app_name}.c)
-    target_link_libraries(${app_name} sysrepo)
+    target_link_libraries(${app_name} sysrepo event)
 endforeach(app_name)
 
 # oven plugin
diff --git a/examples/application_changes_example.c b/examples/application_changes_example.c
index 5c9b6d8bf85a..f5a45a78def8 100644
--- a/examples/application_changes_example.c
+++ b/examples/application_changes_example.c
@@ -20,11 +20,10 @@
 #include <string.h>
 #include <signal.h>
 #include <inttypes.h>
+#include <event.h>
 
 #include "sysrepo.h"
 
-volatile int exit_application = 0;
-
 static void
 print_val(const sr_val_t *value)
 {
@@ -172,28 +171,43 @@ ev_to_str(sr_event_t ev)
     }
 }
 
-static int
-module_change_cb(sr_session_ctx_t *session, const char *module_name, const char *xpath, sr_event_t event,
-        uint32_t request_id, void *private_data)
+struct cb_args {
+    sr_session_ctx_t *session;
+    const char *module_name;
+    const char *xpath;
+    sr_event_t event;
+    uint32_t request_id;
+};
+
+struct subscr_info {
+    struct event_base *evt_base;
+    sr_subscription_ctx_t *subscription;
+    struct cb_args *callback_args;
+    int callback_done;
+    int callback_res;
+};
+
+static void
+async_module_change_cb(evutil_socket_t fd, short what, void *arg)
 {
+    struct subscr_info *info = arg;
+    struct cb_args *args = info->callback_args;
     sr_change_iter_t *it = NULL;
     int rc = SR_ERR_OK;
     sr_change_oper_t oper;
     sr_val_t *old_value = NULL;
     sr_val_t *new_value = NULL;
+    (void)fd;
+    (void)what;
 
-    (void)xpath;
-    (void)request_id;
-    (void)private_data;
+    printf("\n\n ========== EVENT %s CHANGES: ====================================\n\n", ev_to_str(args->event));
 
-    printf("\n\n ========== EVENT %s CHANGES: ====================================\n\n", ev_to_str(event));
-
-    rc = sr_get_changes_iter(session, "//." , &it);
+    rc = sr_get_changes_iter(args->session, "//." , &it);
     if (rc != SR_ERR_OK) {
         goto cleanup;
     }
 
-    while ((rc = sr_get_change_next(session, it, &oper, &old_value, &new_value)) == SR_ERR_OK) {
+    while ((rc = sr_get_change_next(args->session, it, &oper, &old_value, &new_value)) == SR_ERR_OK) {
         print_change(oper, old_value, new_value);
         sr_free_val(old_value);
         sr_free_val(new_value);
@@ -201,22 +215,65 @@ module_change_cb(sr_session_ctx_t *session, const char *module_name, const char
 
     printf("\n ========== END OF CHANGES =======================================");
 
-    if (event == SR_EV_DONE) {
+    if (args->event == SR_EV_DONE) {
         printf("\n\n ========== CONFIG HAS CHANGED, CURRENT RUNNING CONFIG: ==========\n\n");
-        print_current_config(session, module_name);
+        print_current_config(args->session, args->module_name);
     }
 
 cleanup:
     sr_free_change_iter(it);
-    return SR_ERR_OK;
+    info->callback_done = 1;
+    info->callback_res = rc;
+    sr_process_events(info->subscription, NULL, NULL);
+}
+
+static int
+module_change_cb(sr_session_ctx_t *session, const char *module_name, const char *xpath, sr_event_t event,
+        uint32_t request_id, void *private_data)
+{
+    struct subscr_info *info = private_data;
+    struct timeval one_sec = {1, 0};
+
+    if (info->callback_done) {
+        info->callback_done = 0;
+        free(info->callback_args);
+        info->callback_args = NULL;
+        return info->callback_res;
+    }
+
+    info->callback_args = malloc(sizeof(struct cb_args));
+    info->callback_args->session = session;
+    info->callback_args->module_name = module_name;
+    info->callback_args->xpath = xpath;
+    info->callback_args->event = event;
+    info->callback_args->request_id = request_id;
+
+    evtimer_add(evtimer_new(
+        info->evt_base, async_module_change_cb, info), &one_sec);
+
+    return SR_ERR_CALLBACK_SHELVE;
 }
 
 static void
-sigint_handler(int signum)
+signal_cb(evutil_socket_t fd, short what, void *arg)
 {
-    (void)signum;
+    struct subscr_info *info = arg;
+    (void)fd;
+    (void)what;
+    event_base_loopbreak(info->evt_base);
+}
 
-    exit_application = 1;
+static void
+subscription_ready_cb(evutil_socket_t fd, short what, void *arg)
+{
+    struct subscr_info *info = arg;
+    int rc;
+    (void)fd;
+    (void)what;
+    rc = sr_process_events(info->subscription, NULL, NULL);
+    if (rc != SR_ERR_OK) {
+        printf("sr_process_events error: %s\n", sr_strerror(rc));
+    }
 }
 
 int
@@ -226,7 +283,9 @@ main(int argc, char **argv)
     sr_session_ctx_t *session = NULL;
     sr_subscription_ctx_t *subscription = NULL;
     int rc = SR_ERR_OK;
+    int subscription_fd = -1;
     const char *mod_name, *xpath = NULL;
+    struct subscr_info info;
 
     if ((argc < 2) || (argc > 3)) {
         printf("%s <module-to-subscribe> [<xpath-to-subscribe>]\n", argv[0]);
@@ -259,19 +318,31 @@ main(int argc, char **argv)
     print_current_config(session, mod_name);
 
     /* subscribe for changes in running config */
-    rc = sr_module_change_subscribe(session, mod_name, xpath, module_change_cb, NULL, 0, 0, &subscription);
+    rc = sr_module_change_subscribe(
+        session, mod_name, xpath, module_change_cb, &info, 0,
+        SR_SUBSCR_NO_THREAD, &subscription);
     if (rc != SR_ERR_OK) {
         goto cleanup;
     }
 
+    rc = sr_get_event_pipe(subscription, &subscription_fd);
+    if (rc != SR_ERR_OK) {
+        goto cleanup;
+    }
+
+    memset(&info, 0, sizeof(info));
+    info.evt_base = event_base_new();
+    info.subscription = subscription;
+
+    event_add(event_new(
+        info.evt_base, SIGINT, EV_SIGNAL | EV_PERSIST, signal_cb, &info), NULL);
+    event_add(event_new(
+        info.evt_base, subscription_fd, EV_READ | EV_PERSIST, subscription_ready_cb, &info), NULL);
+
     printf("\n\n ========== LISTENING FOR CHANGES ==========\n\n");
 
     /* loop until ctrl-c is pressed / SIGINT is received */
-    signal(SIGINT, sigint_handler);
-    signal(SIGPIPE, SIG_IGN);
-    while (!exit_application) {
-        sleep(1000);
-    }
+    event_base_dispatch(info.evt_base);
 
     printf("Application exit requested, exiting.\n");
 

When running this modified example, we can see that the session cannot be used:

root@6d94624f6003:/tmp/sysrepo-build# ./examples/application_changes_example example /example:conf &
[1] 1199
root@6d94624f6003:/tmp/sysrepo-build# Application will watch for changes in "/example:conf".

 ========== READING RUNNING CONFIG: ==========

/example:conf (container)
/example:conf/system (container)
/example:conf/routing (container)


 ========== LISTENING FOR CHANGES ==========


root@6d94624f6003:/tmp/sysrepo-build# sysrepocfg --import=/home/dev/mgmt/aiosysrepo/examples/example.json -f json -m example


 ========== EVENT change CHANGES: ====================================

[ERR]: Session without changes.
[ERR]: Callback event "change" with ID 1 processing timed out.
sysrepocfg error: Replace config failed (User callback failed)
[1]+  Segmentation fault      (core dumped) ./examples/application_changes_example example /example:conf
root@6d94624f6003:/tmp/sysrepo-build# gdb ./examples/application_changes_example
GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./examples/application_changes_example...done.
(gdb) core-file !tmp!sysrepo-build!examples!application_changes_example.core 
warning: core file may not match specified executable file.
[New LWP 1199]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./examples/application_changes_example example /example:conf'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f7c085102fa in sr_errinfo_free (err_info=0x7ffeec8e4110) at /home/dev/mgmt/sysrepo/src/log.c:281
281                     free((*err_info)->err[i].message);
(gdb) bt
#0  0x00007f7c085102fa in sr_errinfo_free (err_info=0x7ffeec8e4110) at /home/dev/mgmt/sysrepo/src/log.c:281
#1  0x00007f7c0850fb15 in sr_api_ret (session=0x7ffeec8e40f0, err_info=0x5563f984c8b0)
    at /home/dev/mgmt/sysrepo/src/log.c:83
#2  0x00007f7c085010ab in sr_get_changes_iter (session=0x7ffeec8e40f0, xpath=0x5563f93fed47 "//.", 
    iter=0x7ffeec8e4290) at /home/dev/mgmt/sysrepo/src/sysrepo.c:3417
#3  0x00005563f93fe629 in async_module_change_cb (fd=-1, what=1, arg=0x7ffeec8e4430)
    at /home/dev/mgmt/sysrepo/examples/application_changes_example.c:205
#4  0x00007f7c082c0a11 in ?? () from /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6
#5  0x00007f7c082c133f in event_base_loop () from /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6
#6  0x00005563f93feb1b in main (argc=3, argv=0x7ffeec8e4548)
    at /home/dev/mgmt/sysrepo/examples/application_changes_example.c:345

For now, I have worked around this problem by gathering the configuration and changes before scheduling the async callback. However, this is not very practical and ideally, the session should remain usable if the callback returns SR_ERR_CALLBACK_SHELVE.

I'm not sure if that would be possible. And if so, how it would be implemented to avoid stray sessions to remain in the wild if the callback never returns an actual value.

Thanks in advance for your help.

closed time in 2 months

rjarry

issue commentsysrepo/sysrepo

Cannot use implicit session after returning SR_ERR_CALLBACK_SHELVE

Hi Michal,

I understand that it is a complex problem which may require an extensive design modification and/or API change. I think we can live with getting the config/changes inside the callback.

Thanks again for taking the time.

rjarry

comment created time in 2 months

created tagrjarry/libyang-cffi

tagv1.0.130.post3

Python CFFI bindings to libyang

created time in 2 months

push eventrjarry/libyang-cffi

Robin Jarry

commit sha 631eb787d529d0e4db03996d08dc0fbe269c5b41

data: fix create_data_path with booleans When creating a boolean value, it must be converted to string *and* to lower case. Otherwise, libyang fails with the following error: Invalid value "False" in "enabled" element. Add unit test cases to cover this. Fixes: 49fd8ea4dc0d ("Add data tree support") Signed-off-by: Robin Jarry <robin@jarry.cc>

view details

push time in 2 months

issue commentsysrepo/sysrepo

Cannot use implicit session after returning SR_ERR_CALLBACK_SHELVE

By the way, I'm not sure I understand your patch. It looks to me like the new diff field you added to struct sr_change_iter_s is not used in _sr_get_changes_iter nor in sr_get_change_next.

rjarry

comment created time in 2 months

issue commentsysrepo/sysrepo

Cannot use implicit session after returning SR_ERR_CALLBACK_SHELVE

This still means I cannot pass the session pointer to the async callback so that it can fetch the "pending" configuration (not only the changes). Would there be a way of doing this?

rjarry

comment created time in 2 months

more