profile
viewpoint
Eugene Kliuchnikov eustas Magnetosoft Alcatel @Devexperts @google

google/guetzli 12507

Perceptual JPEG encoder

google/butteraugli 1639

butteraugli estimates the psychovisual difference between two images

google/ngx_brotli 1438

NGINX module for Brotli compression

google/pik 755

A new lossy/lossless image format for photos and the internet

google/brunsli 633

Practical JPEG Repacker

eustas/ngx_brotli 204

NGINX module for Brotli compression

eustas/CafeUndZopfli 69

GZip compatible compressor.

eustas/2im 10

Image preview format

google/brotli-wheels 6

Brotli wheels

issue openedgoogle/ngx_brotli

Is this module ready for production?

Seems the module status is under active development.
And there is no official release for this.

So is this module ready for production?

created time in 3 days

issue openedgoogle/brotli

This needs to be reviewed immediately

https://github.com/google/brotli/blob/2a51a85aa86abb4c294c65fab57f3d9c69f10080/js/decode.js#L2151

According to the stated usage of this dictionary, it's a lookup table used only during slow compression. Why would a lookup table need multiple entries of the same thing? To argue that the preceding characters factor in is to argue that cartoonregistrCommonsMuslimsWhat is either an uncompressible string or common enough, and therefore a necessary string to include in this constant. One finds multiple entries of </html> and </script> in this constant, with and without whitespace surrounding.

Aside code repos, most of the uses of this package seem to be detected malware.

I understand this isn't a smoking gun of exploit, and this is a very old piece of code. But could someone familiar with it review the use of this constant? It's included as a dependency in many npm packages.

created time in 4 days

pull request commentgoogle/ngx_brotli

Variable compression level

I added some changes to improve performance when brotli_comp_level is used with a static value. In such cases we don't want to evaluate the complex value on every http request. Instead, evaluation is done at config merge time when nginx is still starting up. Didn't get around to do that commit cleanup, but you can still have a look if you're interested: https://github.com/fholzer/ngx_brotli/commits/variable-compression-level

bartebor

comment created time in 4 days

pull request commentgoogle/ngx_brotli

Variable compression level

This is great! You could set different compression level based on upstream's cache control response header. This would allow you to use better compression for resources that can be cached, and worse compression for no-cache/private resources using a map. I rebased this PR onto master in my fork, though there's still some unrelated commits in my branch which I'd like to clean up first.

bartebor

comment created time in 5 days

issue commentgoogle/brotli

Java source files missing from Github packages, tar.gz and zip, in 1.0.8 and 1.0.9

@eustas Ok, I will look out for the java releases. Thank you!

wltjr

comment created time in 6 days

issue commentDmitryUlyanov/deep-image-prior

Dockerfile does not seem to build successfully

Got it building and working with a couple of minor tweaks. Ill raise a PR but its on this branch:

https://github.com/fibble/deep-image-prior/tree/dockerfile-fixes

seems not fix the problem

entmike

comment created time in 7 days

issue commentgoogle/brotli

Compiler warnings

Microsoft will almost definitely never add support for that. The Visual Studio compiler is a C++ compiler that happens to have some limited support for older versions of C. In C++ there are better, standard ways to do this.

crazydef

comment created time in 7 days

pull request commentgoogle/brotli

Python: use a new output buffer code

FYI, the output buffer code is from here: https://bugs.python.org/issue41486

animalize

comment created time in 7 days

issue commentgoogle/brotli

Compiler warnings

If they're expected, you should be suppressing them.

Taking the following as an example, it would be better (in my opinion) to do something like this anyway:

static BROTLI_INLINE uint32_t BitMask(uint32_t n)
{
#if BROTLI_HAS_UBFX
    /* Masking with this expression turns to a single
       "Unsigned Bit Field Extract" UBFX instruction on ARM. */
    return ~((0xFFFFFFFFu) << n);
#elif BROTLI_HAS_IS_CONSTANT
  if (BROTLI_IS_CONSTANT(n))
  {
    return ~((0xFFFFFFFFu) << n);
  }
  else
#endif
  {
    return kBrotliBitMask[n];
  }
}

We're not so relaxed about suppressing compiler warnings in our code and it would be nice to not have to constantly play catch up when someone includes your headers in a new part of our code.

crazydef

comment created time in 7 days

pull request commentgoogle/brotli

Update

:white_check_mark: Build brotli 1.0.0#1578 completed (commit https://github.com/google/brotli/commit/b37a4131a7 by @)

eustas

comment created time in 7 days

pull request commentgoogle/brotli

Update

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

<!-- need_author_consent -->

eustas

comment created time in 7 days

pull request commentgoogle/brotli

Update

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

<!-- need_author_consent -->

eustas

comment created time in 7 days

issue commentgoogle/ngx_brotli

Nginx docker alpine image

@eustas thanks a lot for your message! Here is working Dockerfile based on it:

# Build
FROM nginx:1.19.6 as build

WORKDIR /build

# install brotli build dependencies
RUN \
    apt-get update > /dev/null \
    && apt-get install gcc git libpcre3-dev libssl-dev make zlib1g-dev -y

# build brotli
RUN git clone https://github.com/google/ngx_brotli.git --recurse-submodules
RUN git clone https://github.com/nginx/nginx.git

WORKDIR /build/nginx

RUN git checkout release-1.19.6
RUN ./auto/configure \
  --add-dynamic-module=/build/ngx_brotli \
  --with-debug \
  --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.19.6/debian/debuild-base/nginx-1.19.6=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'
RUN make modules

# Run
FROM nginx:1.19.6

# add compiled brotli module
COPY --from=build /build/nginx/objs/ngx_http_brotli_filter_module.so /usr/lib/nginx/modules/
COPY --from=build /build/nginx/objs/ngx_http_brotli_static_module.so /usr/lib/nginx/modules/

# run nginx with configuration reload once in every 6 hours
CMD /bin/sh -c 'while :; do /bin/sleep 6h & wait ${!}; /usr/sbin/nginx -s reload; done & /usr/sbin/nginx -g "daemon off;"'
WangHansen

comment created time in 7 days

issue commentgoogle/brotli

Window.BrotliDecode is not set in decode.min.js

@mrpmorris ... There are updates (google/brotli #852) that apparently will fix this (among other things), but those October updates are in limbo. Microsoft is now seeking to drop the use of this script. We can't depend on unmaintained bits.

mrpmorris

comment created time in 8 days

issue openedgoogle/ngx_brotli

CentOS 8, Directadmin with Nginx error installing brotli

My installation but still get some errors.

yum -y install git python python-devel gcc zlib perl libxml2 libxslt autoconf automake bind-utils wget curl unzip gcc-c++ pcre-devel zlib-devel libtool make pam-devel

cd /usr/local/src git clone https://github.com/google/ngx_brotli cd ngx_brotli git submodule update --init

{ result [root@mail ngx_brotli]# git submodule update --init Submodule 'deps/brotli' (https://github.com/google/brotli.git) registered for path 'deps/brotli' Cloning into '/usr/local/src/ngx_brotli/deps/brotli'... Submodule path 'deps/brotli': checked out 'e61745a6b7add50d380cfd7d3883dd6c62fc2c71' }

cd /usr/local/directadmin/custombuild wget https://nginx.org/download/nginx-1.19.6.tar.gz tar -zxvf nginx-1.19.6.tar.gz cd nginx-1.19.6

./configure --with-compat --add-dynamic-module=/usr/local/src/ngx_brotli make modules

nano /etc/nginx/nginx.conf

#mkdir /usr/local/nginx/modules #cp /usr/local/directadmin/custombuild/nginx-1.19.6/objs/ngx_http_brotli_filter_module.so /usr/local/nginx/modules/ngx_http_brotli_filter_module.so #cp /usr/local/directadmin/custombuild/nginx-1.19.6/objs/ngx_http_brotli_static_module.so /usr/local/nginx/modules/ngx_http_brotli_static_module.so

add above the http block

load_module /usr/local/nginx/modules/ngx_http_brotli_filter_module.so; load_module /usr/local/nginx/modules/ngx_http_brotli_static_module.so;

./configure --add-module=/usr/local/src/ngx_brotli make && make install

{ result Configuration summary + using system PCRE library + OpenSSL library is not used + using system zlib library

nginx path prefix: "/usr/local/nginx"
nginx binary file: "/usr/local/nginx/sbin/nginx"
nginx modules path: "/usr/local/nginx/modules"
nginx configuration prefix: "/usr/local/nginx/conf"
nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
nginx pid file: "/usr/local/nginx/logs/nginx.pid"
nginx error log file: "/usr/local/nginx/logs/error.log"
nginx http access log file: "/usr/local/nginx/logs/access.log"
nginx http client request body temporary files: "client_body_temp"
nginx http proxy temporary files: "proxy_temp"
nginx http fastcgi temporary files: "fastcgi_temp"
nginx http uwsgi temporary files: "uwsgi_temp"
nginx http scgi temporary files: "scgi_temp"

}

nginx -t

{ result [root@mail nginx-1.19.6]# nginx -t nginx: [emerg] "load_module" directive is specified too late in /etc/nginx/nginx.conf:14 nginx: configuration file /etc/nginx/nginx.conf test failed }

Error missing files.

mkdir /usr/modules cp /usr/local/directadmin/custombuild/nginx-1.19.6/objs/.so /usr/modules ls /usr/modules chmod 644 /usr/modules/.soso

{ changed positon and fixed the missing files. [root@mail nginx-1.19.6]# nginx -t nginx: [emerg] module "/usr/modules/ngx_http_brotli_filter_module.so" is not binary compatible in /etc/nginx/nginx.conf:10 nginx: configuration file /etc/nginx/nginx.conf test failed }

systemctl reload nginx.service

created time in 8 days

startedeustas/2im

started time in 10 days

issue commentgoogle/brotli

Window.BrotliDecode is not set in decode.min.js

@dsnet ... Hello ... May I ask for a contact at Google in charge of this repo? I'm trying to figure out if this issue will be addressed or won't be addressed ... or perhaps addressed at some point in the distant future.

I'm an Aquent LLC contractor working for Microsoft, and we have an important framework compression feature tied to an official Microsoft documentation suggestion to use this Google Brotli script in its compressed form. However, we can't directly link to a broken script here. We may decide to host our own fixed version of this script (under the license, of course, with proper attribution), or we might decide not to use or refer readers to this script at all. It's hard to make a decision on the best course of action given that I can't get an answer here on the triage of this issue.

Can you put me in touch with the right manager at Google who can help? I can be emailed at llatham at the company (Aquent) dot-com if you want to message me offline. Thanks, Luke

mrpmorris

comment created time in 10 days

issue openedgoogle/brunsli

File signature

Usual JPEG XL files start with bytes ff 0a. As far as I understand, it follows ISOBMFF as other modern ISO formats do. Brunsli files start with 0a 04 42 d2 d5 4e. Why is it so different? If both of these codecs are under the same umbrella name, MIME type and file extension, why they don't share a similar signature?

created time in 11 days

issue commentgoogle/brotli

append to brotli compressed file

The good news, is that we are working on framing format, that (likely) will provide such ability... and much more =) (with some overhead being paid, of course).

Hi, can you please provide issues/plans, where we can read about this?

worenga

comment created time in 11 days

startedeustas/2im

started time in 12 days

issue commentDmitryUlyanov/deep-image-prior

Are masks required for successful restoration?

@abhishekaich27 how do you explain the mask in the loss function? it seems intuitive to me that the mask are 1s for missing pixels because we want to predict those pixels, so the loss function should address that. for those valid pixels, the model do not need to be able to recover anything, thus the loss function does not need to pay attention, so it seem intuitive to filter them out by multiplying 0. I dig into your implementation, it seems that you used mask in an opposite way as i described above. I could not find much more detail about it in the paper, could you please explain the logic? Thanks

This is not my implementation. The author is @DmitryUlyanov !

oh, sorry for miss reference. thanks for response

deeptibhegde

comment created time in 12 days

startedeustas/2im

started time in 12 days

startedeustas/2im

started time in 13 days

startedeustas/2im

started time in 13 days

startedeustas/2im

started time in 13 days

PR opened eustas/2im

Explicitly set image width/height to reduce CLS

This simple change improves the page load time. See https://web.dev/cls/ for more information.

+5 -1

0 comment

1 changed file

pr created time in 13 days

startedeustas/2im

started time in 13 days

fork mathiasbynens/2im

Image preview format

fork in 13 days

issue commentDmitryUlyanov/deep-image-prior

Are masks required for successful restoration?

@abhishekaich27 how do you explain the mask in the loss function?

it seems intuitive to me that the mask are 1s for missing pixels because we want to predict those pixels, so the loss function should address that. for those valid pixels, the model do not need to be able to recover anything, thus the loss function does not need to pay attention, so it seem intuitive to filter them out by multiplying 0.

I dig into your implementation, it seems that you used mask in an opposite way as i described above. I could not find much more detail about it in the paper, could you please explain the logic?

Thanks

This is not my implementation. The author is @DmitryUlyanov !

deeptibhegde

comment created time in 13 days

issue commentDmitryUlyanov/deep-image-prior

Are masks required for successful restoration?

@abhishekaich27 how do you explain the mask in the loss function?

it seems intuitive to me that the mask are 1s for missing pixels because we want to predict those pixels, so the loss function should address that. for those valid pixels, the model do not need to be able to recover anything, thus the loss function does not need to pay attention, so it seem intuitive to filter them out by multiplying 0.

I dig into your implementation, it seems that you used mask in an opposite way as i described above. I could not find much more detail about it in the paper, could you please explain the logic?

Thanks

deeptibhegde

comment created time in 13 days

more