profile
viewpoint

tesla1060/AD3 0

Alternating Directions Dual Decomposition

tesla1060/Algorithmic-Trading-Challenge---Kaggle 0

Algorithmic Trading Challenge implemented as part of the term project for Foundations of Machine Learning at NYU Courant in Fall 2016 (http://cs.nyu.edu/courses/fall16/CSCI-GA.2566-001/index.html/)

tesla1060/arctic 0

High performance datastore for time series and tick data

tesla1060/awesome-lockfree 0

A collection of resources on wait-free and lock-free programming

tesla1060/bitpredict 0

Machine learning for high frequency bitcoin price prediction

tesla1060/Chronicle-Engine 0

A high performance, low latency, reactive processing framework

tesla1060/Chronicle-Map 0

Replicate your Key Value Store across your network, with consistency, persistance and performance.

tesla1060/Chronicle-Queue 0

Micro second messaging that stores everything to disk

tesla1060/courses 0

Course materials for the Data Science Specialization: https://www.coursera.org/specialization/jhudatascience/1

startedphilipperemy/easy-encryption

started time in 3 days

fork tesla1060/pollnet

A collection of non-blocking(polling) network libs for Linux, also support solarflare APIs(Tcpdirect/Efvi)

fork in 17 days

issue commentvincentlaucsb/csv-parser

why DelimWriter constructor parameter is made as a reference?

Hi, @vincentlaucsb my need is actually like below,

#include <iostream>
#include "csv.hpp"
#include <fstream>
#include <map>
#include <vector>
using namespace csv;
using namespace std;

class Recorder {
private:
    std::map<int, csv::CSVWriter<std::ofstream >> csv_writers;
    std::map<int, std::ofstream > ofstream_map;
public:
    Recorder();
};

Recorder::Recorder() {
    vector<int> keys = {1, 2, 3};
    for (auto key:keys) {
        stringstream s;
        s << "test" << key << ".csv";
        ofstream fs(s.str());
        ofstream_map[key] = move(fs);
        csv_writers[key] = csv::make_csv_writer(ofstream_map[key]);
    }
}

int main() {
    Recorder recorder;
    return 0;
}

This does compile,

error C2280: “csv::DelimWriterstd::ofstream,44,34 &csv::DelimWriterstd::ofstream,44,34::operator =(const csv::DelimWriterstd::ofstream,44,34 &)”: 尝试引用已删除的函数

I am wondering what is wrong in my code?

Also this line was picking up from the orginal code of csv.hpp, which seems suppose to construct a DelimWriter out of a filename, but I wonder the same why ifstream is used here.

DelimWriter(const std::string& filename) : DelimWriter(std::ifstream(filename)) {};

The reason I need to use shared_ptr in the map is because the assignment constructor is deleted.

    std::map<int, std::shared_ptr<csv::CSVWriter<std::ofstream >>> csv_writers;
    std::map<int, std::ofstream > ofstream_map;
tesla1060

comment created time in 24 days

issue commentvincentlaucsb/csv-parser

why DelimWriter constructor parameter is made as a reference?

Hi, @vincentlaucsb my need is actually like below,

#include <iostream>
#include "csv.hpp"
#include <fstream>
#include <map>
#include <vector>
using namespace csv;
using namespace std;

class Recorder {
private:
    std::map<int, csv::CSVWriter<std::ofstream >> csv_writers;
    std::map<int, std::ofstream > ofstream_map;
public:
    Recorder();
};

Recorder::Recorder() {
    vector<int> keys = {1, 2, 3};
    for (auto key:keys) {
        stringstream s;
        s << "test" << key << ".csv";
        ofstream fs(s.str());
        ofstream_map[key] = move(fs);
        csv_writers[key] = csv::make_csv_writer(ofstream_map[key]);
    }
}

int main() {
    Recorder recorder;
    return 0;
}

This does compile,

error C2280: “csv::DelimWriterstd::ofstream,44,34 &csv::DelimWriterstd::ofstream,44,34::operator =(const csv::DelimWriterstd::ofstream,44,34 &)”: 尝试引用已删除的函数

I am wondering what is wrong in my code?

Also this line was picking up from the orginal code of csv.hpp, which seems suppose to construct a DelimWriter out of a filename, but I wonder the same why ifstream is used here.

DelimWriter(const std::string& filename) : DelimWriter(std::ifstream(filename)) {};

tesla1060

comment created time in 24 days

issue openedvincentlaucsb/csv-parser

why DelimWriter constructor parameter is made as a reference?

Hi, I understand this is because it has a private member as reference,

    private:
        OutputStream & out;

But design this way, will make below code's ofstream out of scope,

int main() {
    shared_ptr<csv::CSVWriter<std::ofstream>> writer;
    {
        ofstream fs("csv_writer_test.csv");
        writer = make_shared<csv::CSVWriter<std::ofstream>>(fs);
    }

    vector<string> myvector{"d", "e", "f"};
    writer->write_row(move(myvector));
    vector<string> myvector2{"a", "e", "f"};
    writer->write_row(move(myvector2));
    while (true) {}
}

Also is ifstream correct here?

DelimWriter(const std::string& filename) : DelimWriter(std::ifstream(filename)) {};

So what is the right way of construct a DelimWriter with a string?

created time in a month

startedruntimeverification/verified-smart-contracts

started time in 2 months

issue commentgabime/spdlog

add __LINE__ to macro function

@tesla1060 Can you output SPDLOG_ACTIVE_LEVEL ?

+#ifndef SPDLOG_ACTIVE_LEVEL
#define SPDLOG_ACTIVE_LEVEL SPDLOG_LEVEL_TRACE
+#endif
#include <spdlog/spdlog.h>
#include <spdlog/sinks/stdout_sinks.h>

int main(int, char**)
{
+ std::cout << "SPDLOG_ACTIVE_LEVEL: " << SPDLOG_ACTIVE_LEVEL << "\n";
spdlog::set_pattern("[source %s] [function %!] [line %#] %v");
spdlog::set_level(spdlog::level::trace);

SPDLOG_TRACE("This trace"); 
SPDLOG_DEBUG("This is debug..");
SPDLOG_INFO("This is info..");
SPDLOG_ERROR("This is error..");
SPDLOG_CRITICAL("This is critical..");

return 0;
}

apology, I think I have mixed up some testing code, this actually works,

#define SPDLOG_ACTIVE_LEVEL SPDLOG_LEVEL_TRACE
#include <spdlog/spdlog.h>

int main(int, char**)
{

    spdlog::set_pattern("[source %s] [function %!] [line %#] %v");
    spdlog::set_level(spdlog::level::trace);

    SPDLOG_TRACE("This trace");
    SPDLOG_DEBUG("This is debug..");
    SPDLOG_INFO("This is info..");
    SPDLOG_ERROR("This is error..");
    SPDLOG_CRITICAL("This is critical..");

    return 0;
}
MacroGu

comment created time in 2 months

issue commentgabime/spdlog

add __LINE__ to macro function

@tt4g this is what I am getting,

D:\code_test\cmake-build-debug\test_spdlog.exe SPDLOG_ACTIVE_LEVEL: 0 [source spdlog_test.cpp] [function main] [line 16] This trace [source spdlog_test.cpp] [function main] [line 17] This is debug.. [source spdlog_test.cpp] [function main] [line 18] This is info.. [source spdlog_test.cpp] [function main] [line 19] This is error.. [source spdlog_test.cpp] [function main] [line 20] This is critical..

So I should put this on?

spdlog::set_pattern("[source %s] [function %!] [line %#] %v"); spdlog::set_level(spdlog::level::trace);

MacroGu

comment created time in 2 months

startedprogschj/ThreadPool

started time in 2 months

issue commentgabime/spdlog

add __LINE__ to macro function

@tesla1060 What your spdlog version? If using spdlog version before v1.3.0, shoud you define SPDLOG_DEBUG_ON and SPDLOG_TRACE_ON.

https://github.com/gabime/spdlog/blob/10e809cf644d55e5bd7d66d02e2604e2ddd7fb48/include/spdlog/spdlog.h#L129-L132

I am using straight from the master, 1.x, and tried 1.7.0 latest release, both not working, even with

#define SPDLOG_ACTIVE_LEVEL SPDLOG_LEVEL_TRACE
#define SPDLOG_DEBUG_ON
#define SPDLOG_TRACE_ON

I am using vs2019 compiler on windows 10. But I just tried on my linux box, this actually works. A bug on windows?

MacroGu

comment created time in 2 months

issue commentgabime/spdlog

add __LINE__ to macro function

you must

 #define SPDLOG_ACTIVE_LEVEL SPDLOG_LEVEL_TRACE

before including spdlog.h

Hi as what baby-care said, it seems to be the case that even we do,

#define SPDLOG_ACTIVE_LEVEL SPDLOG_LEVEL_TRACE
#include <spdlog/spdlog.h>
#include <spdlog/sinks/stdout_sinks.h>

int main(int, char**)
{

spdlog::set_pattern("[source %s] [function %!] [line %#] %v");
spdlog::set_level(spdlog::level::trace);

SPDLOG_TRACE("This trace"); 
SPDLOG_DEBUG("This is debug..");
SPDLOG_INFO("This is info..");
SPDLOG_ERROR("This is error..");
SPDLOG_CRITICAL("This is critical..");

return 0;
}

I am only getting starting from info showing, trace and debug are missing. wondering why?

MacroGu

comment created time in 2 months

issue openedKjellKod/g3log

how to use g3log as a global variable?

Hi,

Initially, I tried to have a g3log in each class,

     Class A {
        std::unique<g3::LogWorker> logworker =  LogWorker::createLogWorker();
        auto sinkHandle = logworker->addSink(std::make_unique<ConsoleSink>(),
                                             &ConsoleSink::ReceiveLogMessage);
        initializeLogging(logworker.get());
      }


     Class B {
        std::unique<g3::LogWorker> logworker =  LogWorker::createLogWorker();
        auto sinkHandle = logworker->addSink(std::make_unique<ConsoleSink>(),
                                             &ConsoleSink::ReceiveLogMessage);
        initializeLogging(logworker.get());
      }

but this wont work, initialization can only be done in one place.

So I tried global variable approach,

//global.h
extern std::unique_ptr<g3::LogWorker> logworker;
//main.cpp
std::unique_ptr<g3::LogWorker> logworker =  LogWorker::createLogWorker();
auto sinkHandle = logworker->addSink(std::make_unique<ConsoleSink>(),
                                             &ConsoleSink::ReceiveLogMessage);

// initialize the logger before it can receive LOG calls
initializeLogging(logworker.get());
LOG(DEBUG) <<"g3log initialized"; //works fine
//class_a.cpp
#include "global.h"
LOG(DEBUG) <<"some log"; // LOGGER NOT INITIALIZED:

class_a is called after main.cpp why still have LOGGER NOT INITIALIZED error?

Another question is what is the right way of using g3log in different class? Should I use it by passing as a shared_ptr<g3::LogWorker> parameter to each class constructor?

created time in 2 months

issue commentKjellKod/g3log

why g3log seems to running slower than simple write to file, am I using it incorrectly?

  1. If you write with multiple threads to ofstream it’ll just be garbage output (undefined behavior)
  2. You could use your own lock free mechanisms or mutex protected handover to the writing to ofstream.
  3. You could compare with another logger. G3log is not the fastest logger right now. It was when it was made but not anymore. It still stands with having similar performance for each LOG call vs lock-free ones that can have tremendous latency spikes for SOME calls but on average will be faster. Of course the crashhandling of G3log is also nice. There are at least 4-5 lock free, faster, loggers of various quality (some I would absolutely discourage from in case you are working with safety or mission critical software)

With sensible logging (log when it matters not overwhelmingly) ive seen G3log working with 10Gbps line rate packer analyzers and the performance was never an issue. “The fastest” is nice but might come at a price. Figure out what’s acceptable and what features are needed or not needed and make your choice then.

Figure out what’s acceptable in speed. Is acquiring a mutex lock acceptable for a LOG call? The “fastest” is nice but might come with some caveats. I’ve seen G3log used with 10 Gbps line rate network packet analyzers with no issues or dropped packages when doing sensible amount of log calls (even fairly aggressive - but never tested ludicrous amount)

I HAVE debated making a g4log which would use lock free technology but without or minimized latency spikes but it’s simply not enough motivation (zero $) for me to take that on within the foreseeable future.

understand. thanks for the advice.

tesla1060

comment created time in 3 months

issue commentKjellKod/g3log

why g3log seems to running slower than simple write to file, am I using it incorrectly?

I believe ofstream is buffering internally Once in a blue moon it’ll flush it to disk.

Typically you would then have one LOG call that takes a veeery long time but on average it’ll be pretty fast (depending on how frequently it flushes). This is exactly how Googles glog works.

G3log does a handover to a background task. The overhead is essentially the mutex protection to ensure multiple threads can write to the logging sink and ensure FIFO order.

Essentially you are benchmarking a threadsafe struct vs a non-threadsafe struct. The non-threadsafe ofstream doesn’t have to pay for synchronization.

i see, how can I make this benchmark more realistic? my aim is to compare g3log's asynchrnous to synchronous writing to a file?

tesla1060

comment created time in 3 months

issue openedKjellKod/g3log

why g3log seems to running slower than simple write to file, am I using it incorrectly?

Hi, I have performed the following simple benchtest using g3log agains simple fout, the result seems to be slower for g3log, I am wondering if I am using the log the wrong way? As I understand g3log is asynchronous, so each time it is called it pass the log string to another thread for writting to log file?

my simple code as per below,

#include <g3log/g3log.hpp>
#include <g3log/logworker.hpp>
#include <g3log/logmessage.hpp>
#include <chrono>
#include <fstream>
#include <vector>
#include <string>
#include <iostream>
#include <memory>
using namespace std;
using namespace std::chrono;
using namespace g3;

int main() {
    vector<string> str_list;
    int count = 1000000;
    //prepare the string to be written to log
    for (auto i = 0; i < count; i++) {
        string str = "str_" + to_string(i);
        str_list.push_back(str);
    }

    std::shared_ptr<g3::LogWorker> logworker{ g3::LogWorker::createLogWorker() };

    auto default_handle = logworker->addDefaultLogger("log", "./");
    initializeLogging(logworker.get());
    // time g3log
    auto log_start = system_clock::now();
    for (auto i = 0; i < count; i++) {
        LOG(DEBUG) << str_list[i];
    }
    auto log_stop = system_clock::now();
    cout << "g3log:" << duration_cast<std::chrono::milliseconds>(log_stop - log_start).count() << endl;

    ofstream fout;
    fout.open("./file_log.txt", std::ofstream::out);
    // time simple file write
    auto f_start = system_clock::now();
    for (auto i = 0; i < count; i++) {
        fout << str_list[i] << endl;
    }
    auto f_stop = system_clock::now();

    cout <<"simple file:" << duration_cast<std::chrono::milliseconds>(f_stop - f_start).count() << endl;

}

my result is :

g3log:1910 simple file:1252

created time in 3 months

issue commentKjellKod/g3log

linking fails when install in user home directory

I think i have narrow down the problem. gcc 7.2.0 seems to give the linking error

/usr/bin/c++ --version c++ (GCC) 7.2.0

I have another gcc 7.3.0 works fine

/usr/local/bin/c++ --version c++ (GCC) 7.3.0 once i let cmake use gcc 7.3.0, everything works fine.

tesla1060

comment created time in 3 months

startedajtulloch/quantcup-orderbook

started time in 3 months

fork tesla1060/quantcup-orderbook

Fast C++ adaptation of the QuantCup (http://www.quantcup.org/) limit order book.

fork in 3 months

issue commentKjellKod/g3log

linking fails when install in user home directory

No I follow the exact instruction, still not working. this is my cmake -DCMAKE_PREFIX_PATH=/home/myname/opt .. output when cmake the example

cmake -DCMAKE_PREFIX_PATH=/home/myname/opt/ .. -- The C compiler identification is GNU 7.2.0 -- The CXX compiler identification is GNU 7.2.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Found G3LOG: /home/myname/opt/include g3log package: found: TRUE include dir: /home/myname/opt/include libraries: /home/myname/opt/lib64/libg3log.so.2.1.2-119;Threads::Threads -- Configuring done -- Generating done -- Build files have been written to: /home/myname/temp/g3log_example_integration/build

and then when I run cmake --build ., here comes the undefined reference,

Scanning dependencies of target example [ 50%] Building CXX object CMakeFiles/example.dir/example_main.cpp.o [100%] Linking CXX executable example CMakeFiles/example.dir/example_main.cpp.o: In function main': example_main.cpp:(.text+0x279): undefined reference tog3::LogWorker::addDefaultLogger(std::string const&, std::string const&, std::string const&)' example_main.cpp:(.text+0x2e9): undefined reference to g3::FileSink::fileName()' example_main.cpp:(.text+0x329): undefined reference tog3::FileSink::overrideLogDetails(std::string ()(g3::LogMessage const&))' example_main.cpp:(.text+0x33c): undefined reference to g3::LogMessage::FullLogDetailsToString(g3::LogMessage const&)' example_main.cpp:(.text+0x39b): undefined reference tog3::FileSink::overrideLogHeader(std::string const&)' collect2: error: ld returned 1 exit status gmake[2]: *** [example] Error 1 gmake[1]: *** [CMakeFiles/example.dir/all] Error 2 gmake: *** [all] Error 2 [ 50%] Linking CXX executable example CMakeFiles/example.dir/example_main.cpp.o: In function main': example_main.cpp:(.text+0x279): undefined reference tog3::LogWorker::addDefaultLogger(std::string const&, std::string const&, std::string const&)' example_main.cpp:(.text+0x2e9): undefined reference to g3::FileSink::fileName()' example_main.cpp:(.text+0x329): undefined reference tog3::FileSink::overrideLogDetails(std::string ()(g3::LogMessage const&))' example_main.cpp:(.text+0x33c): undefined reference to g3::LogMessage::FullLogDetailsToString(g3::LogMessage const&)' example_main.cpp:(.text+0x39b): undefined reference tog3::FileSink::overrideLogHeader(std::string const&)' collect2: error: ld returned 1 exit status gmake[2]: *** [example] Error 1 gmake[1]: *** [CMakeFiles/example.dir/all] Error 2 gmake: *** [all] Error 2

tesla1060

comment created time in 3 months

fork tesla1060/tardis-python

Python client for tardis.dev - historical tick-level cryptocurrency market data replay API.

https://tardis.dev

fork in 3 months

fork tesla1060/tardis-node

Convenient access to tick-level real-time and historical cryptocurrency market data via Node.js

https://docs.tardis.dev/api/node-js

fork in 3 months

fork tesla1060/tardis-machine

Locally runnable server with built-in data caching, providing both tick-level historical and consolidated real-time cryptocurrency market data via HTTP and WebSocket APIs

https://docs.tardis.dev/api/tardis-machine

fork in 3 months

issue commentKjellKod/g3log

linking fails when install in user home directory

I realize there can be some gotchas when integrating the logger. Thanks for calling this out. I have created an example integration repo that shows how you can do this. Please take a look. https://github.com/KjellKod/g3log_example_integration

strangely, both dynamic and static lib still gives me undefined reference when cmake --build . even though my CMakeList.txt print out the correct lib path

-- Found G3LOG: /home/myname/opt/include g3log package: found: TRUE include dir: /home/myname/opt/include libraries: /home/myname/opt/lib64/libg3log.so.2.1.2-119;Threads::Threads

still undefined reference,

CMakeFiles/test_logger.dir/logger_test.cpp.o: In function CustomSink::ReceiveLogMessage(g3::MoveOnCopy<g3::LogMessage>)': logger_test.cpp:(.text._ZN10CustomSink17ReceiveLogMessageEN2g310MoveOnCopyINS0_10LogMessageEEE[_ZN10CustomSink17ReceiveLogMessageEN2g310MoveOnCopyINS0_10LogMessageEEE]+0xc4): undefined reference tog3::LogMessage::DefaultLogDetailsToString(g3::LogMessage const&)' logger_test.cpp:(.text._ZN10CustomSink17ReceiveLogMessageEN2g310MoveOnCopyINS0_10LogMessageEEE[_ZN10CustomSink17ReceiveLogMessageEN2g310MoveOnCopyINS0_10LogMessageEEE]+0xcc): undefined reference to `g3::LogMessage::toString(std::string (*)(g3::LogMessage const&)) const' collect2: error: ld returned 1 exit status gmake[2]: *** [test_logger] Error 1 gmake[1]: *** [CMakeFiles/test_logger.dir/all] Error 2 gmake: *** [all] Error 2

tesla1060

comment created time in 3 months

issue openedKjellKod/g3log

linking fails when install in user home directory

Describe the bug I have successfully compiled the lib following below, because I have no permission on /usr/local, I am having a linking error but exact code can build on a root previlidged machine by install lib in /usr/local

Below are the details on the error

cmake .. -DCMAKE_INSTALL_PREFIX=/home/myname/opt
make 
make install

This has successfully installed lib in /home/myname/opt/lib64

ls /home/myname/opt/lib64/ cmake libg3logger.so libg3logger.so.1.3.0-0

my CMakeList.txt can find the lib with no problem, and I can print out {G3LOG_LIBRARIES}

# CMakeList.txt
cmake_minimum_required (VERSION 3.11)
project(test)

set(g3logger_DIR "/home/myname/opt/lib64/cmake/g3logger")
find_package(g3logger CONFIG REQUIRED)
message(${G3LOG_INCLUDE_DIRS})
message(${G3LOG_LIBRARIES})
include_directories(${G3LOG_INCLUDE_DIRS})

if (CMAKE_BUILD_TYPE MATCHES Debug)
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++17 -O0 -g -ggdb -Wall -pthread")
elseif (CMAKE_BUILD_TYPE MATCHES Release)
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++17 -O3 -Wall -pthread")
else()
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++17 -O3 -Wall -pthread")
endif()
add_executable(test_logger logger_test.cpp)
target_link_libraries(test_logger g3logger)

make VERBOSE=1 gives the following output,

make VERBOSE=1 /usr/bin/cmake3 -H/home/myname/test -B/home/myname/test/build --check-build-system CMakeFiles/Makefile.cmake 0 /usr/bin/cmake3 -E cmake_progress_start /home/myname/test/build/CMakeFiles /home/myname/test/build/CMakeFiles/progress.marks make -f CMakeFiles/Makefile2 all make[1]: Entering directory /home/myname/test/build' make -f CMakeFiles/test_logger.dir/build.make CMakeFiles/test_logger.dir/depend make[2]: Entering directory/home/myname/test/build' cd /home/myname/test/build && /usr/bin/cmake3 -E cmake_depends "Unix Makefiles" /home/myname/test /home/myname/test /home/myname/test/build /home/myname/test/build /home/myname/test/build/CMakeFiles/test_logger.dir/DependInfo.cmake --color= Dependee "/home/myname/test/build/CMakeFiles/test_logger.dir/DependInfo.cmake" is newer than depender "/home/myname/test/build/CMakeFiles/test_logger.dir/depend.internal". Dependee "/home/myname/test/build/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/home/myname/test/build/CMakeFiles/test_logger.dir/depend.internal". Scanning dependencies of target test_logger make[2]: Leaving directory /home/myname/test/build' make -f CMakeFiles/test_logger.dir/build.make CMakeFiles/test_logger.dir/build make[2]: Entering directory/home/myname/test/build' [ 50%] Building CXX object CMakeFiles/test_logger.dir/logger_test.cpp.o /usr/bin/c++ -isystem /home/myname/opt/include -std=c++17 -O3 -Wall -pthread -std=gnu++14 -o CMakeFiles/test_logger.dir/logger_test.cpp.o -c /home/myname/test/logger_test.cpp [100%] Linking CXX executable test_logger /usr/bin/cmake3 -E cmake_link_script CMakeFiles/test_logger.dir/link.txt --verbose=1 /usr/bin/c++ -std=c++17 -O3 -Wall -pthread CMakeFiles/test_logger.dir/logger_test.cpp.o -o test_logger -Wl,-rpath,/home/myname/opt/lib64 /home/myname/opt/lib64/libg3logger.so.1.3.0-0 -lpthread CMakeFiles/test_logger.dir/logger_test.cpp.o: In function CustomSink::ReceiveLogMessage(g3::MoveOnCopy<g3::LogMessage>)': logger_test.cpp:(.text._ZN10CustomSink17ReceiveLogMessageEN2g310MoveOnCopyINS0_10LogMessageEEE[_ZN10CustomSink17ReceiveLogMessageEN2g310MoveOnCopyINS0_10LogMessageEEE]+0xc4): undefined reference tog3::LogMessage::DefaultLogDetailsToString(g3::LogMessage const&)' logger_test.cpp:(.text._ZN10CustomSink17ReceiveLogMessageEN2g310MoveOnCopyINS0_10LogMessageEEE[_ZN10CustomSink17ReceiveLogMessageEN2g310MoveOnCopyINS0_10LogMessageEEE]+0xcc): undefined reference to g3::LogMessage::toString(std::string (*)(g3::LogMessage const&)) const' collect2: error: ld returned 1 exit status make[2]: *** [test_logger] Error 1 make[2]: Leaving directory/home/myname/test/build' make[1]: *** [CMakeFiles/test_logger.dir/all] Error 2 make[1]: Leaving directory `/home/myname/test/build' make: *** [all] Error 2

created time in 3 months

issue openedTencent/rapidjson

Should a document be reused in repeated parsing?

Hi, I have a callback function in a class looks like below,

void myclass::on_message(const string& msg) {
    Document doc;
    doc.Parse(msg);
}

or should I declare doc as a member of myclass and use as

class myclass {
...
Document doc;
}

void myclass::on_message(const string& msg) {
    doc.Parse(msg);
}

I was wondering if the first method will be less performant because each callback will construct a Document object? or I can actually dont care because construction of Document object is pretty fast? what is the best way to handle such scenario?

If I go for second, I fould this thread https://github.com/Tencent/rapidjson/issues/1605, mentioned,

d.SetNull();
d.GetAllocator().Clear();

But after a few loops, it throws rapidjson::kParseErrorValueInvalid if I am testing using doc.HasMember("table") which should be valid, because it runs well without the above two lines of clear code,ust that doc.GetAllocator().Capacity() keeps growing.

created time in 3 months

startedOneflow-Inc/oneflow

started time in 3 months

issue commentmachinezone/IXWebSocket

Is it possible to make select in a busy wait loop to improve latency?

Hi bsergean, for trading purpose, the main concern is not throughput, but latency. It is better to reduce the reaction time once data reached the socket, so as the stackoverflow post mentioned,

When timeout is NULL then if there's no data your process will be put to sleep in the WAIT_INTERRUPTIBLE state waiting for data to become available. This incurs a penalty of at least two context switches, one away from your process and one back to it when data becomes available

It will be better to avoid context switch as much as possible.

I have read up the code a little bit, just would like to clarify,

IXSocket.cpp
const int Socket::kDefaultPollNoTimeout = -1; // No poll timeout by default
const int Socket::kDefaultPollTimeout = kDefaultPollNoTimeout;

I take this -1 means null, what if I set this to 0, so that poll will be doing a busy wait, is that going to be some problem?

tesla1060

comment created time in 3 months

issue openedmachinezone/IXWebSocket

Is it possible to make select in a busy wait loop to improve latency?

The docs mentioned:

No manual polling to fetch data is required. Data is sent and received instantly by using a background thread for receiving data and the select system call to be notified by the OS of incoming data. No timeout is used for select so that the background thread is only woken up when data is available, to optimize battery life

Because I am using the websocket for crypto trading, so I am ok with a cpu perpetually polling the socket, and according to this thread, busy polling seems to improve latecny by more than two times, though I did not test my self,

https://stackoverflow.com/questions/13937096/linux-select-performance-scenarios

I was wondering is it possible to set polling in the code? which part should be modified? and possibily a configuration in the future release?

Also my use case is receiving a lot of market data from the socket, but occasionally send out order to the crypto exchange, and I will be using a cloud server, so I dont have constraint on number of thread as well as cpu cycle. I was wondering is there any design in IXWebSocket that is optimized for smart phone, but may not suite for my use case well? Thanks.

created time in 3 months

issue commentmachinezone/IXWebSocket

Is it safe to send in background thread after some receive?

thanks. got it.

tesla1060

comment created time in 3 months

more