profile
viewpoint

ysmolsky/cryptopals-matasano-solutions 12

Go solutions for cryptopals challenges: http://cryptopals.com/

ysmolsky/unnamed-archery-game 9

WIP of semi archery arcade 2D shooter

ysmolsky/clojure-df 2

Clone of Dwarf Fortress on Clojure

ysmolsky/beanstalk 0

Go client for beanstalkd

ysmolsky/ct 0

(Relatively) Easy Unit Testing in C

ysmolsky/emacs.d 0

my settings

ysmolsky/flycheck 0

On the fly syntax checking for GNU Emacs

ysmolsky/go-lazyfoo-sdl 0

LazyFoo' SDL lessons in Go

ysmolsky/go-sdl2 0

SDL2 binding for Go

ysmolsky/gopl 0

Solutions for Go Programming Language Book

push eventbeanstalkd/go-beanstalk

Tim Cooper

commit sha a8ec173d4fb5a435cd82169af9952e05c855da2b

implement Unwrap methods on error types (#40)

view details

push time in 15 hours

issue closedbeanstalkd/go-beanstalk

named tubeset issue

I was trying to place something in a named tubeset but it seems to only go in the default tubeset.

conn, err := beanstalk.Dial("tcp", "127.0.0.1:11300")

if err != nil {
	...
}

tubeSet := beanstalk.NewTubeSet(conn, "email")

m := // some struct that marshals into JSON

jobbytes, err := json.Marshal(m)

if err != nil {
	....
}

jobid, err := tubeSet.Conn.Tube.Put(jobbytes, 1, 0, 120*time.Second)

Then when I try to pull the job out in a different client

tubeSet := beanstalk.NewTubeSet(conn, "email")

id, body, err := tubeSet.Reserve(time.Duration(10) * time.Second)

I get a "reserve-with-timeout: timeout" error

but if I switch to just doing the Reserve call on the connection using the "default" tubeset, it pulls the data out without a problem.

How do I go about using a different tubeset than the default? Could you show me an example?

closed time in 15 hours

tvmaly

issue commentbeanstalkd/go-beanstalk

named tubeset issue

@bontibon thank you for the answer here. Closing as non-issue.

tvmaly

comment created time in 15 hours

PR closed beanstalkd/go-beanstalk

Fix example import path.

Importing github.com/kr/beanstalk in example_test.go will create indirect dependency for github.com/beanstalkd/go-beanstalk.

Rename github.com/kr/beanstalk into github.com/beanstalkd/go-beanstalk.

please kindly review @kr @sergeyklay.

+2 -1

2 comments

1 changed file

rolandhawk

pr closed time in 15 hours

pull request commentbeanstalkd/go-beanstalk

Fix example import path.

It's a dupe. Closing

rolandhawk

comment created time in 15 hours

issue closedbeanstalkd/beanstalkd

beanstalkd down because of free() invalid size.

Hi, I'v got this errors on syslog and process killed, could you explain it to me?!

Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: *** Error in `/usr/bin/beanstalkd': free(): invalid size: 0x00005600f0fd3020 ***
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: ======= Backtrace: =========
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: /lib/x86_64-linux-gnu/libc.so.6(+0x70bfb)[0x7f59eecf3bfb]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: /lib/x86_64-linux-gnu/libc.so.6(+0x76fc6)[0x7f59eecf9fc6]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: /lib/x86_64-linux-gnu/libc.so.6(+0x7780e)[0x7f59eecfa80e]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: /usr/bin/beanstalkd(+0x6472)[0x5600f03be472]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: /usr/bin/beanstalkd(+0x7918)[0x5600f03bf918]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: /usr/bin/beanstalkd(+0x82dd)[0x5600f03c02dd]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: /usr/bin/beanstalkd(+0x1a24)[0x5600f03b9a24]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f59eeca32e1]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: /usr/bin/beanstalkd(+0x1caa)[0x5600f03b9caa]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: ======= Memory map: ========
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 5600f03b8000-5600f03c5000 r-xp 00000000 fe:01 29209                      /usr/bin/beanstalkd
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 5600f05c5000-5600f05c6000 r--p 0000d000 fe:01 29209                      /usr/bin/beanstalkd
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 5600f05c6000-5600f05c7000 rw-p 0000e000 fe:01 29209                      /usr/bin/beanstalkd
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 5600f05c7000-5600f05df000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 5600f0fcc000-56019f539000 rw-p 00000000 00:00 0                          [heap]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59e4000000-7f59e4021000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59e4021000-7f59e8000000 ---p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59e995d000-7f59eb95e000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ed64c000-7f59ed662000 r-xp 00000000 fe:01 4180                       /lib/x86_64-linux-gnu/libgcc_s.so.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ed662000-7f59ed861000 ---p 00016000 fe:01 4180                       /lib/x86_64-linux-gnu/libgcc_s.so.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ed861000-7f59ed862000 r--p 00015000 fe:01 4180                       /lib/x86_64-linux-gnu/libgcc_s.so.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ed862000-7f59ed863000 rw-p 00016000 fe:01 4180                       /lib/x86_64-linux-gnu/libgcc_s.so.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ed863000-7f59ed876000 r-xp 00000000 fe:01 1094                       /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ed876000-7f59eda75000 ---p 00013000 fe:01 1094                       /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eda75000-7f59eda76000 r--p 00012000 fe:01 1094                       /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eda76000-7f59eda77000 rw-p 00013000 fe:01 1094                       /lib/x86_64-linux-gnu/libgpg-error.so.0.21.0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eda77000-7f59eda7a000 r-xp 00000000 fe:01 11911                      /lib/x86_64-linux-gnu/libdl-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eda7a000-7f59edc79000 ---p 00003000 fe:01 11911                      /lib/x86_64-linux-gnu/libdl-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59edc79000-7f59edc7a000 r--p 00002000 fe:01 11911                      /lib/x86_64-linux-gnu/libdl-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59edc7a000-7f59edc7b000 rw-p 00003000 fe:01 11911                      /lib/x86_64-linux-gnu/libdl-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59edc7b000-7f59edced000 r-xp 00000000 fe:01 753                        /lib/x86_64-linux-gnu/libpcre.so.3.13.3
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59edced000-7f59edeec000 ---p 00072000 fe:01 753                        /lib/x86_64-linux-gnu/libpcre.so.3.13.3
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59edeec000-7f59edeed000 r--p 00071000 fe:01 753                        /lib/x86_64-linux-gnu/libpcre.so.3.13.3
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59edeed000-7f59edeee000 rw-p 00072000 fe:01 753                        /lib/x86_64-linux-gnu/libpcre.so.3.13.3
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59edeee000-7f59edf06000 r-xp 00000000 fe:01 11928                      /lib/x86_64-linux-gnu/libpthread-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59edf06000-7f59ee105000 ---p 00018000 fe:01 11928                      /lib/x86_64-linux-gnu/libpthread-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee105000-7f59ee106000 r--p 00017000 fe:01 11928                      /lib/x86_64-linux-gnu/libpthread-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee106000-7f59ee107000 rw-p 00018000 fe:01 11928                      /lib/x86_64-linux-gnu/libpthread-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee107000-7f59ee10b000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee10b000-7f59ee212000 r-xp 00000000 fe:01 3688                       /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee212000-7f59ee412000 ---p 00107000 fe:01 3688                       /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee412000-7f59ee414000 r--p 00107000 fe:01 3688                       /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee414000-7f59ee41b000 rw-p 00109000 fe:01 3688                       /lib/x86_64-linux-gnu/libgcrypt.so.20.1.6
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee41b000-7f59ee42c000 r-xp 00000000 fe:01 1168                       /usr/lib/x86_64-linux-gnu/liblz4.so.1.7.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee42c000-7f59ee62b000 ---p 00011000 fe:01 1168                       /usr/lib/x86_64-linux-gnu/liblz4.so.1.7.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee62b000-7f59ee62c000 r--p 00010000 fe:01 1168                       /usr/lib/x86_64-linux-gnu/liblz4.so.1.7.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee62c000-7f59ee62d000 rw-p 00011000 fe:01 1168                       /usr/lib/x86_64-linux-gnu/liblz4.so.1.7.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee62d000-7f59ee652000 r-xp 00000000 fe:01 778                        /lib/x86_64-linux-gnu/liblzma.so.5.2.2
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee652000-7f59ee851000 ---p 00025000 fe:01 778                        /lib/x86_64-linux-gnu/liblzma.so.5.2.2
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee851000-7f59ee852000 r--p 00024000 fe:01 778                        /lib/x86_64-linux-gnu/liblzma.so.5.2.2
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee852000-7f59ee853000 rw-p 00025000 fe:01 778                        /lib/x86_64-linux-gnu/liblzma.so.5.2.2
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee853000-7f59ee85a000 r-xp 00000000 fe:01 11930                      /lib/x86_64-linux-gnu/librt-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ee85a000-7f59eea59000 ---p 00007000 fe:01 11930                      /lib/x86_64-linux-gnu/librt-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eea59000-7f59eea5a000 r--p 00006000 fe:01 11930                      /lib/x86_64-linux-gnu/librt-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eea5a000-7f59eea5b000 rw-p 00007000 fe:01 11930                      /lib/x86_64-linux-gnu/librt-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eea5b000-7f59eea80000 r-xp 00000000 fe:01 2681                       /lib/x86_64-linux-gnu/libselinux.so.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eea80000-7f59eec7f000 ---p 00025000 fe:01 2681                       /lib/x86_64-linux-gnu/libselinux.so.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eec7f000-7f59eec80000 r--p 00024000 fe:01 2681                       /lib/x86_64-linux-gnu/libselinux.so.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eec80000-7f59eec81000 rw-p 00025000 fe:01 2681                       /lib/x86_64-linux-gnu/libselinux.so.1
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eec81000-7f59eec83000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eec83000-7f59eee18000 r-xp 00000000 fe:01 11908                      /lib/x86_64-linux-gnu/libc-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59eee18000-7f59ef018000 ---p 00195000 fe:01 11908                      /lib/x86_64-linux-gnu/libc-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef018000-7f59ef01c000 r--p 00195000 fe:01 11908                      /lib/x86_64-linux-gnu/libc-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef01c000-7f59ef01e000 rw-p 00199000 fe:01 11908                      /lib/x86_64-linux-gnu/libc-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef01e000-7f59ef022000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef022000-7f59ef045000 r-xp 00000000 fe:01 11904                      /lib/x86_64-linux-gnu/ld-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef1aa000-7f59ef1b1000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef1b1000-7f59ef235000 r-xp 00000000 fe:01 6640                       /lib/x86_64-linux-gnu/libsystemd.so.0.17.0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef235000-7f59ef236000 ---p 00084000 fe:01 6640                       /lib/x86_64-linux-gnu/libsystemd.so.0.17.0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef236000-7f59ef239000 r--p 00084000 fe:01 6640                       /lib/x86_64-linux-gnu/libsystemd.so.0.17.0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef239000-7f59ef23a000 rw-p 00087000 fe:01 6640                       /lib/x86_64-linux-gnu/libsystemd.so.0.17.0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef23a000-7f59ef23b000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef241000-7f59ef245000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef245000-7f59ef246000 r--p 00023000 fe:01 11904                      /lib/x86_64-linux-gnu/ld-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef246000-7f59ef247000 rw-p 00024000 fe:01 11904                      /lib/x86_64-linux-gnu/ld-2.24.so
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7f59ef247000-7f59ef248000 rw-p 00000000 00:00 0
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7ffc379f3000-7ffc37a14000 rw-p 00000000 00:00 0                          [stack]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7ffc37be5000-7ffc37be7000 r--p 00000000 00:00 0                          [vvar]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: 7ffc37be7000-7ffc37be9000 r-xp 00000000 00:00 0                          [vdso]
Aug 16 11:11:20 nl-kjg-queue-ba beanstalkd[617]: ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aug 16 11:11:20 nl-kjg-queue-ba systemd[1]: beanstalkd.service: Main process exited, code=killed, status=6/ABRT
Aug 16 11:11:20 nl-kjg-queue-ba systemd[1]: beanstalkd.service: Unit entered failed state.
Aug 16 11:11:20 nl-kjg-queue-ba systemd[1]: beanstalkd.service: Failed with result 'signal'.

closed time in 10 days

honarkhah

issue commentbeanstalkd/beanstalkd

Beanstalk not purging old binlogs?

Nope, I deploy using docker container in which I build it from scratch.

laszlof

comment created time in a month

issue commentbeanstalkd/beanstalkd

Beanstalk not purging old binlogs?

One note from old user of 1.10. During many years we never had this problem, but when we migrated to the different OS (AWS-something) it happened a couple of times. Upgrade to 1.11 has solved the issue.

laszlof

comment created time in a month

pull request commentbeanstalkd/beanstalkd

[WIP] Added memcheck to the build matrix

I think, we won't go there yet. I have somewhat reduced interest in this project since autumn has started. I was a little depressed that there is very little support by the community. What was done is good enough for my work and I will fix only critical bugs. Most likely I will not try to hunt down all kind of memory leaks since this software is intended as crash-only and there is no sense in graceful exit.

sergeyklay

comment created time in 2 months

issue closedbeanstalkd/beanstalkd

Degradation on many tubes

I have tried to create about 1500 tubes and than run 300 instances of simpliest php code.

$connection = new \Pheanstalk\Pheanstalk(
    'host',
    11300,
    5,
    false
);


while (true) {
$queue = 'test' . rand(1,2000);
    $job = $connection
        ->watch($queue)
        ->ignore('default')
        ->reserve();


    echo microtime(true) . ' PUT START'. PHP_EOL;

$before = microtime(true);


    $connection->putInTube(
        'test' . rand(1, 2000),
        $job->getData()
    );
$after = microtime(true);
$delta = $after - $before;
echo 'DELTA = ' . $delta . PHP_EOL;

    $connection->delete($job);
} 

Results was bad: 1 instance: 1567547165.0457 PUT START string(20) "INSERTED 820713869 " DELTA = 0.0016741752624512 1567547165.0474 PUT FINISH

300 instances 1567547210.9205 PUT START string(20) "INSERTED 820886790 " DELTA = 0.021309852600098 1567547210.9419 PUT FINISH

Time for inserting increased more than 10 times.

Can anyone suggest where can be the problem.

closed time in 2 months

bessudnov

issue commentbeanstalkd/beanstalkd

Degradation on many tubes

Closing as not actionable issue. I won't fix this because it requires architecture overhaul.

bessudnov

comment created time in 2 months

issue closedbeanstalkd/beanstalkd

beanstalkd[2776] general protection ip:401f74 sp:7ffc346076d0 error:0 in beanstalkd[400000+e000]

beanstalk 1.10

the start cmd: beanstalkd -b /data/beanstalkd/binlog -F -z 65535000

the Error: beanstalkd[2776] general protection ip:401f74 sp:7ffc346076d0 error:0 in beanstalkd[400000+e000]

how to fixed the bug

closed time in 2 months

jingwu15

issue commentbeanstalkd/beanstalkd

beanstalkd[2776] general protection ip:401f74 sp:7ffc346076d0 error:0 in beanstalkd[400000+e000]

I am sorry, but that version was released long time ago and we do not support it anymore. Please use more recent release or even master branch to run this server.

jingwu15

comment created time in 2 months

push eventbeanstalkd/beanstalkd

alingse

commit sha 9576b6f2271231a93069beb9b2afecd375d32de0

add delay ttr maximum value to chinese doc

view details

push time in 2 months

PR merged beanstalkd/beanstalkd

add delay ttr maximum value

https://github.com/beanstalkd/beanstalkd/blob/master/doc/protocol.txt#L137-L139

https://github.com/beanstalkd/beanstalkd/blob/master/doc/protocol.txt#L146

+2 -2

1 comment

1 changed file

alingse

pr closed time in 2 months

delete branch beanstalkd/beanstalkd

delete branch : add-buffer

delete time in 2 months

PR closed beanstalkd/beanstalkd

Reviewers
read from socket to the buffer instead of a char

~ % benchstat old.txt new.txt name old time/op new time/op delta _put_delete_1k 318µs ± 3% 260µs ± 3% -18.38% (p=0.000 n=9+8) _put_delete_64k 574µs ± 2% 514µs ± 8% -10.49% (p=0.000 n=10+10) _put_delete_8 306µs ± 1% 256µs ± 2% -16.51% (p=0.000 n=8+9) _put_delete_8k 365µs ± 3% 306µs ± 5% -16.10% (p=0.000 n=9+10)

name old speed new speed delta _put_delete_1k 4.83MB/s ± 0% 4.97MB/s ± 1% +2.94% (p=0.000 n=9+9) _put_delete_64k 129MB/s ± 0% 148MB/s ±28% +14.36% (p=0.000 n=10+10) _put_delete_8 40.0kB/s ± 0% 40.0kB/s ± 0% ~ (all equal) _put_delete_8k 28.4MB/s ±34% 38.9MB/s ± 1% +36.76% (p=0.000 n=10+10)

Updates #430

+56 -25

10 comments

1 changed file

ysmolsky

pr closed time in 2 months

issue closedbeanstalkd/beanstalkd

beanstalkd progress exit

this morning I use command sudo /usr/local/beanstalkd/bin/beanstalkd -b /data/beanstalkd -l 0.0.0.0 -p 11300 -u beanstalkd & start service,but in afternoon,the progress exit. beanstalkd version is 1.11,os version is centos 7.6.1810. I not found any relate message in /var/log/message

closed time in 4 months

pz9042

issue closedbeanstalkd/beanstalkd

make error linuxmint 19.1

-localhost ~/Downloads/beanstalkd $ make check
cc -Wall -Werror -Wformat=2 -g   -c -o testheap.o testheap.c
In file included from testheap.c:1:0:
/usr/lib/gcc/x86_64-linux-gnu/7/include/stdint.h:9:16: fatal error: stdint.h: No such file or directory
 # include_next <stdint.h>
                ^~~~~~~~~~
compilation terminated.
<builtin>: recipe for target 'testheap.o' failed
make: *** [testheap.o] Error 1

closed time in 4 months

akwatra

issue commentbeanstalkd/beanstalkd

make error linuxmint 19.1

Something is wrong with your compiler suite.

akwatra

comment created time in 4 months

issue commentbeanstalkd/beanstalkd

beanstalkd progress exit

This is a normal message, it means that client's connection was lost during the read operation.

pz9042

comment created time in 5 months

issue commentbeanstalkd/beanstalkd

Degradation on many tubes

@bessudnov did any of the above help you in some way?

bessudnov

comment created time in 5 months

issue commentbeanstalkd/beanstalkd

beanstalkd progress exit

By default beanstalkd writes into stderr and not to /var/log/messages. You might want to capture stderr.

pz9042

comment created time in 5 months

issue commentbeanstalkd/beanstalkd

beanstalkd progress exit

@pz9042 can you provide more details?

pz9042

comment created time in 5 months

issue closedbeanstalkd/beanstalkd

prot.c: null dereference in enqueue_reserved_jobs

I'm using Infer to analise the code of the latest version of Beanstalkd. The tool found the following error:

pointer `j` last assigned on line 509 could be null and is dereferenced by call to `enqueue_job()` at line 510, column 13.
  508.       while (job_list_any_p(&c->reserved_jobs)) {
  509.           j = job_remove(c->reserved_jobs.next);
  510. >         r = enqueue_job(c->srv, j, 0, 0);
  511.           if (r < 1) bury_job(c->srv, j, 0);
  512.           global_stat.reserved_ct--;

This error occurs because the value of the job_remove function is not checked before it is used, and it can return NULL in two situations: 1 - When the argument passed to the job_remove is NULL; 2 - When the value returned by the job_list_any_p is NULL. Therefore, the value of the variable j must be checked before being passed to the enqueue_job function. In the enqueue_job function, a dereference is made of j, so if j is NULL this error will cause undefined behavior of the program.

closed time in 6 months

PatriciaSVMonteiro

issue commentbeanstalkd/beanstalkd

prot.c: null dereference in enqueue_reserved_jobs

Job cannot be NULL when passed to enqueue_job because job_remove won't return NULL because job_list_any_p has returned non-NULL value. This code could be more simple I agree and right now it looks like so:

void
enqueue_reserved_jobs(Conn *c)
{
    while (!job_list_is_empty(&c->reserved_jobs)) {
        Job *j = job_list_remove(c->reserved_jobs.next);
        int r = enqueue_job(c->srv, j, 0, 0);
        if (r < 1)
            bury_job(c->srv, j, 0);
        global_stat.reserved_ct--;
        j->tube->stat.reserved_ct--;
        c->soonest_job = NULL;
    }
}

The code for double-linked lists is hard to reason about in general.

PatriciaSVMonteiro

comment created time in 6 months

issue commentbeanstalkd/beanstalkd

running out of open files on server causes client connections to hang

Setting this via rlimit seems like a sane solution.

ramSeraph

comment created time in 6 months

issue commentbeanstalkd/beanstalkd

add a "copy" command to copy/clone the job into multiple tubes

I have set the wrong label, since this feature is not decided upon (yet).

MartinWickman

comment created time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha 0fcf38b1782b10cbdb2ee03a79a401812c612663

introduce the bool data type This is to improve readability of some functions: is_job_reserved_by_conn, touch_job, and is_valid_tube.

view details

push time in 6 months

delete branch beanstalkd/beanstalkd

delete branch : code-style

delete time in 6 months

PR merged beanstalkd/beanstalkd

introduce the bool data type

This is to improve readability of some functions: is_job_reserved_by_conn, touch_job, and is_valid_tube.

+28 -23

1 comment

2 changed files

ysmolsky

pr closed time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha d18aa2341538c04f4da03176792056289dc20f35

fix response for the command that is too long Before the patch, if a client sent too long command, server would respond by BAD_FORMAT, followed by UNKNOWN_COMMAND. This resulted in desync between client and server. This patch adds another state STATE_WANT_ENDLINE, whose purpose is to drop command line until the EOL is found. Client can send as big command as he want and server will be able to skip it and return only one error message: BAD_FORMAT. Some STATE_* constants were renamed to improve readability. Fixes #337

view details

Yury Smolsky

commit sha 88870638299bab04b4435b6a8aeed879b0c70a38

add the "reserve-job" command A job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this command does not differ from reservations made by "reserve" or "reserve-with-timeout" commands. The new command does not produce "deadline soon" when there are other jobs about to expire. Fixes #310 Fixes #222 Fixes #240

view details

Yury Smolsky

commit sha 474268c418dc6403e06fb7c1b5d438068f56ce42

introduce the bool data type This is to improve readability of some functions: is_job_reserved_by_conn, touch_job, and is_valid_tube.

view details

push time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha 88870638299bab04b4435b6a8aeed879b0c70a38

add the "reserve-job" command A job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this command does not differ from reservations made by "reserve" or "reserve-with-timeout" commands. The new command does not produce "deadline soon" when there are other jobs about to expire. Fixes #310 Fixes #222 Fixes #240

view details

push time in 6 months

delete branch beanstalkd/beanstalkd

delete branch : reserve-job

delete time in 6 months

PR merged beanstalkd/beanstalkd

add the "reserve-job" command

A job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this command does not differ from reservations made by "reserve" or "reserve-with-timeout" commands. The new command does not produce "deadline soon" when there are other jobs about to expire.

Fixes #310 Fixes #222 Fixes #240

+269 -25

1 comment

4 changed files

ysmolsky

pr closed time in 6 months

issue closedbeanstalkd/beanstalkd

add a "move" command

From: https://groups.google.com/forum/#!searchin/beanstalk-talk/bury|sort:date/beanstalk-talk/AyZhb4e6i24/5WKPuEEvTNkJ

This would be really useful to make routing jobs between tubes safer and more efficient. I'm imagining a "move" operation for a reserved job. It would be very similar to "release", but instead of returning the job to the tube from which it was reserved, it would move the job to the end of the currently used tube.

Documentation:

The "move" command moves a reserved job into the ready queue of the currently used tube. It is normally used when routing jobs between tubes. It looks like this:

move id pri delay \r\n

  • id is the job id to move.
  • pri is a new priority to assign to the job.
  • delay is a new integer number of seconds to wait before putting the job in the ready queue. The job will be in the "delayed" state during this time.

The client expects one line of response, which may be:

  • MOVED\r\n to indicate success.
  • BURIED\r\n if the server ran out of memory trying to grow the priority queue data structure.
  • NOT_FOUND\r\nif the job does not exist or is not reserved by the client.

closed time in 6 months

MartinWickman

issue closedbeanstalkd/beanstalkd

support reserving a specific job by id

Idea:

reserve-job <id>

Output:

RESERVED <id> <bytes>\r\n
<data>\r\n

or

ALREADY_RESERVED\r\n

or

NOT_FOUND\r\n

This would fix #109 and #20 and lots of other cases where one would like to modify a job. The ALREADY_RESERVED would make sure that noone is processing the job, before I modify it.

I don't know the code (yet), would it be somewhat efficient to execute reserve-job <id>?

closed time in 6 months

JensRantil

issue closedbeanstalkd/beanstalkd

delay any existing job by id

Is that possible to delay any existing job? As per my research, there's no any direct facility to delay any existing job.

I think, to delay any existing job, you need to delete previous one and add new one with delay information. But, in that case I'm unable to find actual tube name from which job was fetched. That also unable to get from reserve command response.

Correct me if I'm wrong.

Thank...

closed time in 6 months

sanjaymohnani

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha 8ad7a87a64c6fe1f8d0d1683f5f887773e7dc4b6

add the "reserve-job" command A job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this command does not differ from reservations made by "reserve" or "reserve-with-timeout" commands. The new command does not produce "deadline soon" when there are other jobs about to expire. Fixes #310 Fixes #222 Fixes #240

view details

push time in 6 months

issue commentbeanstalkd/beanstalkd

proposal: off-load job bodies to WAL

This is too complex to implement in the code. File handling code is not ideal, and implementing this feature would make it even more harder. Memory is cheap, and I do not think that converting beanstalkd into some kind of database is a good thing.

JensRantil

comment created time in 6 months

issue closedbeanstalkd/beanstalkd

return only one response, UNKNOWN_COMMAND, to the command longer than LINE_BUF_SIZE

So far, I've seen two situations where the server and the client can become out of sync:

  1. The client sends a command longer than LINE_BUF_SIZE.
  2. The client sends a syntactically wrong put command with a job payload.

There maybe more. Right now the server reports the error as soon as it detects it and keeps going. But the connection stream is then out of sync for an undefined number of invalid command cycles. Wouldn't be more appropriate just to punish the client and disconnect it?

closed time in 6 months

etanol
IssuesEvent

issue commentbeanstalkd/beanstalkd

a command without parameters should not yield `UNKNOWN_COMMAND`

Closed by mistake

JensRantil

comment created time in 6 months

Pull request review commentbeanstalkd/beanstalkd

add the "reserve-job" command

 dispatch_cmd(Conn *c)         process_queue();         return; +    case OP_RESERVE_JOB:+        if (read_u64(&id, c->cmd + CMD_RESERVE_JOB_LEN, NULL)) {+            reply_msg(c, MSG_BAD_FORMAT);+            return;+        }+        op_ct[type]++;++        // This command could produce "deadline soon" if+        // the connection has a reservation about to expire.+        // But we choose not to do it, until a good reason comes up.

Done. Also I have created an entry in protocol.txt.

ysmolsky

comment created time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha eee8116a7e96f987c711729182932b3af1742c3e

add the "reserve-job" command A job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this command does not differ from reservations made by "reserve" or "reserve-with-timeout" commands. The new command does not produce "deadline soon" when there are other jobs about to expire. Fixes #310 Fixes #222 Fixes #240

view details

push time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha d18aa2341538c04f4da03176792056289dc20f35

fix response for the command that is too long Before the patch, if a client sent too long command, server would respond by BAD_FORMAT, followed by UNKNOWN_COMMAND. This resulted in desync between client and server. This patch adds another state STATE_WANT_ENDLINE, whose purpose is to drop command line until the EOL is found. Client can send as big command as he want and server will be able to skip it and return only one error message: BAD_FORMAT. Some STATE_* constants were renamed to improve readability. Fixes #337

view details

Yury Smolsky

commit sha 99c79469bd596ba88e38d7cc80c77fb3f3206dfa

add the "reserve-job" command Job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this command does not differ from reservations made by "reserve" or "reserve-with-timeout" commands. The new command does not produce "deadline soon" when there are other jobs about to expire. Fixes #310 Fixes #222 Fixes #240

view details

push time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha d18aa2341538c04f4da03176792056289dc20f35

fix response for the command that is too long Before the patch, if a client sent too long command, server would respond by BAD_FORMAT, followed by UNKNOWN_COMMAND. This resulted in desync between client and server. This patch adds another state STATE_WANT_ENDLINE, whose purpose is to drop command line until the EOL is found. Client can send as big command as he want and server will be able to skip it and return only one error message: BAD_FORMAT. Some STATE_* constants were renamed to improve readability. Fixes #337

view details

push time in 6 months

delete branch beanstalkd/beanstalkd

delete branch : too-long-cmd

delete time in 6 months

PR merged beanstalkd/beanstalkd

Reviewers
fix response for the command that is too long

Before the patch, if a client sent too long command, server would respond by BAD_FORMAT, followed by UNKNOWN_COMMAND. This resulted in desync between client and server.

This patch adds another state STATE_WANT_ENDLINE, whose purpose is to drop command line until the EOL is found. Client can send as big command as he want and server will be able to skip it and return only one error message: BAD_FORMAT.

Some STATE_* constants were renamed to improve readability.

Fixes #337

+77 -44

4 comments

3 changed files

ysmolsky

pr closed time in 6 months

issue closedbeanstalkd/beanstalkd

a command without parameters should not yield `UNKNOWN_COMMAND`

put
UNKNOWN_COMMAND

If I execute put without arguments, I get UNKNOWN_COMMAND. Since put is a valid command, but I am missing parameters, I am expecting BAD_FORMAT. As soon as I add a single parameter I get BAD_FORMAT:

put 0
BAD_FORMAT

closed time in 6 months

JensRantil

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha 438d88ec485af1c3c80dba7eb1f817befc576d61

add the "reserve-job" command Job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this commands does not differ from reservations made by "reserve" or "reserve-with_timeout" commands. The new command does not produce "deadline soon" when there are other jobs are about to expire. Documenation is TBD. Fixes #310 Fixes #222 Fixes #240

view details

push time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha 8dfb13fe9ff8faa3bce8b0bd3a47d70cda47df64

add the "reserve-job" command Job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this commands does not differ from reservations made by "reserve" or "reserve-with_timeout" commands. The new command does not produce "deadline soon" when there are other jobs are about to expire. Documenation is TBD. Fixes #310 Fixes #222 Fixes #240

view details

push time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha d76e71f096f2b5074d3fd6b77b2da2f27733bb22

add the "reserve-job" command Job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this commands does not differ from reservations made by "reserve" or "reserve-with_timeout" commands. The new command does not produce "deadline soon" when there are other jobs are about to expire. Documenation is TBD. Fixes #310 Fixes #222 Fixes #240

view details

push time in 6 months

PR opened beanstalkd/beanstalkd

introduce the bool data type

This is to improve readability of some functions: is_job_reserved_by_conn, touch_job, and is_valid_tube.

+28 -23

0 comment

2 changed files

pr created time in 6 months

create barnchbeanstalkd/beanstalkd

branch : code-style

created branch time in 6 months

PR opened beanstalkd/beanstalkd

Reviewers
add the "reserve-job" command

Job can be reserved by an ID in the Ready, Buried or Delayed states. Reservation made with this commands does not differ from reservations made by "reserve" or "reserve-with_timeout" commands. The new command does not produce "deadline soon" when there are other jobs are about to expire.

Documenation is TBD.

Fixes #310 Fixes #222 Fixes #240

+244 -23

0 comment

3 changed files

pr created time in 6 months

create barnchbeanstalkd/beanstalkd

branch : reserve-job

created branch time in 6 months

issue commentbeanstalkd/beanstalkd

support reserving a specific job by id

I am on it.

JensRantil

comment created time in 6 months

pull request commentbeanstalkd/beanstalkd

fix response for the command that is too long

I have fixed the protocol regarding why BAD_FORMAT may be returned.

ysmolsky

comment created time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha fae6722d656d6d1c90c2483bfb4af6d352f937c9

remove the -c and -n flags that control log compaction Log compaction should not be switched off. Manuals were fixed to mention those flags as having no effect. Fixed #552

view details

Yury Smolsky

commit sha 8b95c051bd25622f6cc1f4aeefe55462bfcc042b

fix response for the command that is too long Before the patch, if a client sent too long command, server would respond by BAD_FORMAT, followed by UNKNOWN_COMMAND. This resulted in desync between client and server. This patch adds another state STATE_WANT_ENDLINE, whose purpose is to drop command line until the EOL is found. Client can send as big command as he want and server will be able to skip it and return only one error message: BAD_FORMAT. Some STATE_* constants were renamed to improve readability. Fixes #337

view details

push time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha 25725daf8dc58f41a8b1e425b91b68ff6c0a0781

fix response for the command that is too long Before the patch, if a client sent too long command, server would respond by BAD_FORMAT, followed by UNKNOWN_COMMAND. This resulted in desync between client and server. This patch adds another state STATE_WANT_ENDLINE, whose purpose is to drop command line until the EOL is found. Client can send as big command as he want and server will be able to skip it and return only one error message: BAD_FORMAT. Some STATE_* constants were renamed to improve readability. Fixes #337

view details

push time in 6 months

pull request commentbeanstalkd/beanstalkd

fix response for the command that is too long

@ysmolsky Should we update docs as well?

Docs mention that already, but expanding it a little would not hurt I guess. https://github.com/beanstalkd/beanstalkd/blob/master/doc/protocol.txt#L44

ysmolsky

comment created time in 6 months

issue commentbeanstalkd/beanstalkd

Degradation on many tubes

Reproducing this in our benchmarks. I have injected this code to bench_put_delete_size just before actual benchmark. It watches for T amount of tubes. It does not put jobs in them. Everything happens in the default tube.

    for (i = 0; i < T; i++) {
        sprintf(buf, "watch tube%d\r\n", i + 1);
        mustsend(fd, buf);
        ckrespsub(fd, "WATCHING ");
    }

T=0:

ctbench_put_delete_0008	    5000	    335256 ns/op	   0.04 MB/s
ctbench_put_delete_1024	    3000	    338097 ns/op	   3.07 MB/s
ctbench_put_delete_8192	    3000	    378445 ns/op	  24.25 MB/s

T=1000:

ctbench_put_delete_0008	    5000	    392244 ns/op	   0.04 MB/s
ctbench_put_delete_1024	    5000	    390163 ns/op	   4.68 MB/s
ctbench_put_delete_8192	    3000	    423477 ns/op	  23.93 MB/s

T=10000:

ctbench_put_delete_0008	    2000	    916187 ns/op	   0.01 MB/s
ctbench_put_delete_1024	    2000	    910175 ns/op	   1.89 MB/s
ctbench_put_delete_8192	    2000	    855391 ns/op	  15.30 MB/s

Then instead of watching tubes in the connection where benchmarking is performed, I have created another connection where T tubes were just watched. T=1000:

ctbench_put_delete_0008	    5000	    392381 ns/op	   0.04 MB/s
ctbench_put_delete_1024	    5000	    393842 ns/op	   4.67 MB/s
ctbench_put_delete_8192	    3000	    431670 ns/op	  23.87 MB/s

T=10000:

ctbench_put_delete_0008	    2000	    957047 ns/op	   0.01 MB/s
ctbench_put_delete_1024	    2000	    910631 ns/op	   1.89 MB/s
ctbench_put_delete_8192	    2000	    867341 ns/op	  15.26 MB/s

This proves the point that amount of tubes affects regular put/reserver operations. Performance is reduced: 1.2 times for 1k tubes, 3 times for 10k tubes.

bessudnov

comment created time in 6 months

PR opened beanstalkd/beanstalkd

Reviewers
fix response for the command that is too long

Before the patch, if a client sent too long command, server would respond by BAD_FORMAT, followed by UNKNOWN_COMMAND. This resulted in desync between client and server.

This patch adds another state STATE_WANT_ENDLINE, whose purpose is to drop command line until the EOL is found. Client can send as big command as he want and server will be able to skip it and return only one error message: BAD_FORMAT.

Some STATE_* constants were renamed to improve readability.

Fixes #337

+75 -43

0 comment

2 changed files

pr created time in 6 months

push eventbeanstalkd/beanstalkd

Yury Smolsky

commit sha cc008c0c4e07a3df40f21da78c20a89bd2a3886d

fix response for the command that is too long Before the patch, if a client sent too long command, server would respond by BAD_FORMAT, followed by UNKNOWN_COMMAND. This resulted in desync between client and server. This patch adds another state STATE_WANT_ENDLINE, whose purpose is to drop command line until the EOL is found. Client can send as big command as he want and server will be able to skip it and return only one error message: BAD_FORMAT. Some STATE_* constants were renamed to improve readability. Fixes #337

view details

push time in 6 months

create barnchbeanstalkd/beanstalkd

branch : too-long-cmd

created branch time in 6 months

issue commentbeanstalkd/beanstalkd

Degradation on many tubes

It is not only performance of tube_find is degraded in case of big amount of tubes. soonest_delayed_job is affected as well, as it scans through all the tubes. And next_awaited job: it picks the job with smallest priority among all awaited jobs in all tubes. next_awaited job is called by process_queue, and that is called when job is added and reserved too.

Amount of tubes affects performance of these functions linearly.

bessudnov

comment created time in 6 months

issue commentbeanstalkd/beanstalkd

Degradation on many tubes

This what I understood from our conversation in Twitter: author uses > 10k tubes, each worker connects, "watch" some tube, "reserve" the job, disconnects. In that case "tube_find" might be the bottleneck.

bessudnov

comment created time in 6 months

pull request commentbeanstalkd/beanstalkd

[WIP] Added memcheck to the build matrix

@ysmolsky I'm sorry for the delay (too busy at work). Can this wait for some (short) time? I'll try to investigate a bit more in the near future.

Sure thing. By all means, no rushing from me.

sergeyklay

comment created time in 6 months

more