Access your Docker for Mac's internal networks from your macOS host machine
Infrastructure for creating Dockerfiles for package building
Shell helper that automatically sets and unsets environment variables
OpenERP account_bankimport fork with better MT940 support
Create and run an independent local PostgreSQL database for your Django project
Go generic implementation of the logr interface
hackerbeerstest/hackerbeerstest 1
Yep, this is only for testing purposes. No fancy code. Sorry.
Python wrapper for GitHub API v3
PR opened mna/redisc
Changes the testserver to use the testing.TB interface instead of *testing.T to allow use in benchmarks.
This should not cause any issues for existing users of this package.
pr created time in 12 days
Pull request review commentPowerDNS/pdns
dnsdist: Add support for Lua per-thread FFI rules and actions
class LuaFFIAction: public DNSAction func_t d_func; }; +class LuaFFIPerThreadAction: public DNSAction+{+public:+ typedef std::function<int(dnsdist_ffi_dnsquestion_t* dq)> func_t;++ LuaFFIPerThreadAction(const std::string& code): d_functionCode(code), d_functionID(s_functionsCounter++)+ {+ }++ DNSAction::Action operator()(DNSQuestion* dq, std::string* ruleresult) const override+ {+ try {+ auto& state = t_perThreadStates[d_functionID];+ if (!state.d_initialized) {+ setupLuaFFIPerThreadContext(state.d_luaContext);+ state.d_func = state.d_luaContext.executeCode<func_t>(d_functionCode);+ state.d_initialized = true;+ }++ dnsdist_ffi_dnsquestion_t dqffi(dq);+ auto ret = state.d_func(&dqffi);+ if (ruleresult) {+ if (dqffi.result) {+ *ruleresult = *dqffi.result;+ }+ else {+ // default to empty string+ ruleresult->clear();+ }+ }+ return static_cast<DNSAction::Action>(ret);+ } catch (const std::exception &e) {+ warnlog("LuaFFIPerThreadAction failed inside Lua, returning ServFail: %s", e.what());+ } catch (...) {+ warnlog("LuaFFIPerthreadAction failed inside Lua, returning ServFail: [unknown exception]");+ }+ return DNSAction::Action::ServFail;+ }++ string toString() const override+ {+ return "Lua FFI per-thread script";+ }++private:+ struct PerThreadState+ {+ LuaContext d_luaContext;+ func_t d_func;+ bool d_initialized{false};+ };+ static uint64_t s_functionsCounter;
Great, then everything should be fine. Thanks!
comment created time in a month
Pull request review commentPowerDNS/pdns
dnsdist: Add support for Lua per-thread FFI rules and actions
class LuaFFIAction: public DNSAction func_t d_func; }; +class LuaFFIPerThreadAction: public DNSAction+{+public:+ typedef std::function<int(dnsdist_ffi_dnsquestion_t* dq)> func_t;++ LuaFFIPerThreadAction(const std::string& code): d_functionCode(code), d_functionID(s_functionsCounter++)+ {+ }++ DNSAction::Action operator()(DNSQuestion* dq, std::string* ruleresult) const override+ {+ try {+ auto& state = t_perThreadStates[d_functionID];+ if (!state.d_initialized) {+ setupLuaFFIPerThreadContext(state.d_luaContext);+ state.d_func = state.d_luaContext.executeCode<func_t>(d_functionCode);+ state.d_initialized = true;+ }++ dnsdist_ffi_dnsquestion_t dqffi(dq);+ auto ret = state.d_func(&dqffi);+ if (ruleresult) {+ if (dqffi.result) {+ *ruleresult = *dqffi.result;+ }+ else {+ // default to empty string+ ruleresult->clear();+ }+ }+ return static_cast<DNSAction::Action>(ret);+ } catch (const std::exception &e) {+ warnlog("LuaFFIPerThreadAction failed inside Lua, returning ServFail: %s", e.what());+ } catch (...) {+ warnlog("LuaFFIPerthreadAction failed inside Lua, returning ServFail: [unknown exception]");+ }+ return DNSAction::Action::ServFail;+ }++ string toString() const override+ {+ return "Lua FFI per-thread script";+ }++private:+ struct PerThreadState+ {+ LuaContext d_luaContext;+ func_t d_func;+ bool d_initialized{false};+ };+ static uint64_t s_functionsCounter;
Assuming the Lua state is also destroyed, I wonder if this would also allow Lua to garbage collect all the objects, in which case there should be no issue.
comment created time in a month
Pull request review commentPowerDNS/pdns
dnsdist: Add support for Lua per-thread FFI rules and actions
class LuaFFIAction: public DNSAction func_t d_func; }; +class LuaFFIPerThreadAction: public DNSAction+{+public:+ typedef std::function<int(dnsdist_ffi_dnsquestion_t* dq)> func_t;++ LuaFFIPerThreadAction(const std::string& code): d_functionCode(code), d_functionID(s_functionsCounter++)+ {+ }++ DNSAction::Action operator()(DNSQuestion* dq, std::string* ruleresult) const override+ {+ try {+ auto& state = t_perThreadStates[d_functionID];+ if (!state.d_initialized) {+ setupLuaFFIPerThreadContext(state.d_luaContext);+ state.d_func = state.d_luaContext.executeCode<func_t>(d_functionCode);+ state.d_initialized = true;+ }++ dnsdist_ffi_dnsquestion_t dqffi(dq);+ auto ret = state.d_func(&dqffi);+ if (ruleresult) {+ if (dqffi.result) {+ *ruleresult = *dqffi.result;+ }+ else {+ // default to empty string+ ruleresult->clear();+ }+ }+ return static_cast<DNSAction::Action>(ret);+ } catch (const std::exception &e) {+ warnlog("LuaFFIPerThreadAction failed inside Lua, returning ServFail: %s", e.what());+ } catch (...) {+ warnlog("LuaFFIPerthreadAction failed inside Lua, returning ServFail: [unknown exception]");+ }+ return DNSAction::Action::ServFail;+ }++ string toString() const override+ {+ return "Lua FFI per-thread script";+ }++private:+ struct PerThreadState+ {+ LuaContext d_luaContext;+ func_t d_func;+ bool d_initialized{false};+ };+ static uint64_t s_functionsCounter;
Are threads ever removed? If they are, one good solution would be to push the Lua state to a pool for later reuse by a new thread.
My main concern is that every state may open a new LMDB env with a large mapsize.
comment created time in a month
Pull request review commentPowerDNS/pdns
dnsdist: Add support for Lua per-thread FFI rules and actions
class LuaFFIAction: public DNSAction func_t d_func; }; +class LuaFFIPerThreadAction: public DNSAction+{+public:+ typedef std::function<int(dnsdist_ffi_dnsquestion_t* dq)> func_t;++ LuaFFIPerThreadAction(const std::string& code): d_functionCode(code), d_functionID(s_functionsCounter++)+ {+ }++ DNSAction::Action operator()(DNSQuestion* dq, std::string* ruleresult) const override+ {+ try {+ auto& state = t_perThreadStates[d_functionID];+ if (!state.d_initialized) {+ setupLuaFFIPerThreadContext(state.d_luaContext);+ state.d_func = state.d_luaContext.executeCode<func_t>(d_functionCode);+ state.d_initialized = true;+ }++ dnsdist_ffi_dnsquestion_t dqffi(dq);+ auto ret = state.d_func(&dqffi);+ if (ruleresult) {+ if (dqffi.result) {+ *ruleresult = *dqffi.result;+ }+ else {+ // default to empty string+ ruleresult->clear();+ }+ }+ return static_cast<DNSAction::Action>(ret);+ } catch (const std::exception &e) {+ warnlog("LuaFFIPerThreadAction failed inside Lua, returning ServFail: %s", e.what());+ } catch (...) {+ warnlog("LuaFFIPerthreadAction failed inside Lua, returning ServFail: [unknown exception]");+ }+ return DNSAction::Action::ServFail;+ }++ string toString() const override+ {+ return "Lua FFI per-thread script";+ }++private:+ struct PerThreadState+ {+ LuaContext d_luaContext;+ func_t d_func;+ bool d_initialized{false};+ };+ static uint64_t s_functionsCounter;
Related question: what is the life cycle of the created Lua states? Do I understand correctly that one is created globally for every thread and never destroyed? The reason I am asking is that we may need a way to do cleanup if they get destroyed frequently.
comment created time in a month
pull request commentPowerDNS/pdns
Rec: cumulative and Prometheus friendly histograms
Before I document this, I would like confirmation that the current output (see above) is as Prometheus likes. @wojas, can you confirm?
Note that for dnsdist we do some basic checking using
promtool check metrics1.
I just ran the example in the PR description through promtool check metrics after adding the following comments to them to tell promtool that these are histograms:
# HELP pdns_recursor_cumul_answers_seconds Histogram of stuff
# TYPE pdns_recursor_cumul_answers_seconds histogram
# HELP pdns_recursor_cumul_auth4answers_seconds Histogram of stuff
# TYPE pdns_recursor_cumul_auth4answers_seconds histogram
# HELP pdns_recursor_cumul_auth6answers_seconds Histogram of stuff
# TYPE pdns_recursor_cumul_auth6answers_seconds histogram
These comments are not required and I have yet to see any tool that actually depends on them, but this allowed promtool to actually check it as a histogram. I did notice that promtool does not do very strict validation.
comment created time in a month
push eventPowerDNS/pdns-builder
commit sha 008993828b290c54ece190ea12e82d32520db576
Reproducible RPM builds Add support for reproducible builds in RHEL 8+ and derived distributions: - Pass a SOURCE_DATE_EPOCH build arg to Docker. - Set the defines needed for reproducible RPMs. - Add BUILDER_SOURCE_DATE_FROM_SPEC_MTIME to enable the spec mtime as the SOURCE_DATE_EPOCH for vendor builds. - Test reproducible builds in CI.
commit sha 67f5d25ae6872bae6988437e35bb8a34aa4f160b
Update usage text for -e
commit sha b7d12ad7f67b058faf11d322ccdf510da9f9d5eb
Merge pull request #48 from PowerDNS/source-date-epoch Reproducible RPM builds
push time in a month
PR merged PowerDNS/pdns-builder
Reproducible RPM builds and SOURCE_DATE_EPOCH env var.
Note that this only works for RHEL 8+ and derived distributions, as the RPM version in RHEL 7 does not support reproducible builds.
Progress
- [x] Pass
SOURCE_DATE_EPOCHenv var as a build arg (up to the Dockerfiles to explicitly use it) - [x] Set
SOURCE_DATE_EPOCHto last commit time if not set - [x] Define magic variables to kindly ask rpmbuild for reproducible RPM builds
- [ ] Discuss: is it OK to always set these? Perhaps only if
SOURCE_DATE_EPOCHis set?
- [ ] Discuss: is it OK to always set these? Perhaps only if
- [x] Example of args to pass to tar in demo
- [x] Add centos-8 as demo target
- [x] Upgrade alpine
- [x] Also make it work for the vendor RPM without breaking the Docker cache layers by using the last commit time
- [x] CI test reproducible RPM build using centos-8 target (does not work with centos-7)
- [x] Add a few words on this to the README
- [x] Cleanup PR when everything works
pr closed time in a month
push eventPowerDNS/pdns-builder
commit sha 67f5d25ae6872bae6988437e35bb8a34aa4f160b
Update usage text for -e
push time in a month
Pull request review commentPowerDNS/pdns-builder
usage() { echo " -V VERSION - Override version (default: run gen-version)" echo " -R RELEASE - Override release tag (default: '1pdns', do not include %{dist} here)" echo " -m MODULES - Build only specific components (comma separated; warning: this disables install tests)"- echo " -e EPOCH - Set a specific Epoch for packages"+ echo " -e EPOCH - Set a specific Epoch for RPM packages"
Like this?
echo " -e EPOCH - Set a specific version Epoch for RPM/DEB packages"
comment created time in a month
Pull request review commentPowerDNS/pdns-builder
usage() { echo " -V VERSION - Override version (default: run gen-version)" echo " -R RELEASE - Override release tag (default: '1pdns', do not include %{dist} here)" echo " -m MODULES - Build only specific components (comma separated; warning: this disables install tests)"- echo " -e EPOCH - Set a specific Epoch for packages"+ echo " -e EPOCH - Set a specific Epoch for RPM packages"
OK, I will change it. I just wanted to make clear it has nothing to do with SOURCE_DATE_EPOCH.
How about something like this?
echo " -e EPOCH - Set a specific Epoch for RPM/DEB packages, to reset the package versioning"
comment created time in a month
pull request commentPowerDNS/pdns-builder
Updated the readme and squashed the commits. No further changes planned.
comment created time in a month
push eventPowerDNS/pdns-builder
commit sha 008993828b290c54ece190ea12e82d32520db576
Reproducible RPM builds Add support for reproducible builds in RHEL 8+ and derived distributions: - Pass a SOURCE_DATE_EPOCH build arg to Docker. - Set the defines needed for reproducible RPMs. - Add BUILDER_SOURCE_DATE_FROM_SPEC_MTIME to enable the spec mtime as the SOURCE_DATE_EPOCH for vendor builds. - Test reproducible builds in CI.
push time in a month
push eventPowerDNS/pdns-builder
commit sha df00d2328b2437d6cde12d7161fdeb1b964f8dcb
Remove touch again, changes version
push time in a month
push eventPowerDNS/pdns-builder
commit sha aecb0d11d27d60c2503506d5f31ade38121f89a6
Tests: invalidate build context just in case
push time in a month
Pull request review commentPowerDNS/pdns-builder
for spec in "${specs[@]}"; do # Build the rpm and record which files are new rpm_file_list > /tmp/rpms-before- rpmbuild --define "_sdistdir /sdist" -ba "$spec"+ rpmbuild \+ --define "_sdistdir /sdist" \+ --define "_buildhost reproducible" \+ --define "source_date_epoch_from_changelog Y" \
The answer is here: https://fossies.org/linux/rpm/build/build.c#l_298
298 if (rpmExpandNumeric("%{?source_date_epoch_from_changelog}") &&
299 getenv("SOURCE_DATE_EPOCH") == NULL) {
300 /* Use date of first (== latest) changelog entry */
The SOURCE_DATE_EPOCH env var always overrides the changelog timestamp. This is not what I expected, but still safe to keep in the script.
comment created time in a month
push eventPowerDNS/pdns-builder
commit sha 6455890ab182b0dc8740a808ce284be4d90d42b1
Add BUILDER_SOURCE_DATE_FROM_SPEC_MTIME BUILDER_SOURCE_DATE_FROM_SPEC_MTIME can be set to use the mtime of the spec file as the SOURCE_DATE_EPOCH. This is useful for vendor packages.
push time in a month
Pull request review commentPowerDNS/pdns
dnsdist: Add support for range-based lookups into a Key-Value store
std::vector<std::string> KeyValueLookupKeySourceIP::getKeys(const ComboAddress& std::vector<std::string> result; ComboAddress truncated(addr); + std::string key; if (truncated.isIPv4()) { truncated.truncate(d_v4Mask);- result.emplace_back(reinterpret_cast<const char*>(&truncated.sin4.sin_addr.s_addr), sizeof(truncated.sin4.sin_addr.s_addr));+ key.reserve(sizeof(truncated.sin4.sin_addr.s_addr) + (d_includePort ? sizeof(truncated.sin4.sin_port) : 0));+ key.append(reinterpret_cast<const char*>(&truncated.sin4.sin_addr.s_addr), sizeof(truncated.sin4.sin_addr.s_addr)); } else if (truncated.isIPv6()) { truncated.truncate(d_v6Mask);- result.emplace_back(reinterpret_cast<const char*>(&truncated.sin6.sin6_addr.s6_addr), sizeof(truncated.sin6.sin6_addr.s6_addr));+ key.reserve(sizeof(truncated.sin6.sin6_addr.s6_addr) + (d_includePort ? sizeof(truncated.sin4.sin_port) : 0));
The ports are only useful for IPv4 CG-NAT setups, not for IPv6. And I guess sin4.sin_port would need to use sin6?
comment created time in a month