profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/curtisliu/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

Chromega/ggj2020 1

Global Game Jam 2020

curtisliu/hunt-tools 1

Tools for Mystery Hunt

curtisliu/aws-xray-sdk-java 0

The official AWS X-Ray Recorder SDK for Java.

pull request commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

fix the error in build?

dantetam

comment created time in 5 hours

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

+//+//  Amplitude.m+//  Copyright (c) 2013 Amplitude Inc. (https://amplitude.com/)

same question for the year here. And do we plan to add some test to cover this file?

dantetam

comment created time in 5 hours

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

+//+//  Amplitude.m+//  Copyright (c) 2013 Amplitude Inc. (https://amplitude.com/)

Is this year should be up to date?

dantetam

comment created time in 5 hours

push eventamplitude/Amplitude-ReactNative

Gustavo Genovese

commit sha 86036692b4d32d202585b3c028390400ddc81cc5

Added support for tvOS

view details

push time in 12 hours

issue closedamplitude/Amplitude-JavaScript

Is there any possibility the device id to be duplicated?

Summary

The duplicated device id is observed in my service.

  • total: almost 100,000 device ids
  • duplicated: almost 400 device ids

Question 1

Is there any possibility the device id to be duplicated?

Question 2

If Q1 yes, to avoid the situation, does I have to use the re-generate method of amplitude's instance? -> amplitude.getInstance().regenerateDeviceId()

closed time in 19 hours

jason-hwang

issue commentamplitude/Amplitude-JavaScript

Is there any possibility the device id to be duplicated?

Thank you for the answer @kelvin-lu

Sorry for my mistake, I've realized there was a problem with my service logic. I didn't consider the users who are using multi-login in their one device. I will make a new issue If I have any questions.

Great thanks again 😄

jason-hwang

comment created time in 19 hours

push eventamplitude/Amplitude-iOS

Dante Tam

commit sha a52d34fb142d3b927ef4ef50cd5766ecdef8b096

Add copyright header for Amplitude

view details

push time in 2 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

+//

Oh sorry, that's the wrong header. I didn't realize you were talking about a copyright

dantetam

comment created time in 2 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

+//

Why is is resolved? Amplitude copyright is needed for all new files.

dantetam

comment created time in 2 days

push eventamplitude/Amplitude-iOS

Dante Tam

commit sha 3bffb3b26671dd16eccf804a1f773d0d8f29c9e6

Fix error case of unclosed JSON (no ending bracket), try finishing the file and do it again

view details

push time in 2 days

issue commentamplitude/Amplitude-Java

Trying to create a simple Java example, part 2

Updates: We added the IMPORTANT NOTE to the developer center for this issue.
Thanks for letting us know the potential problem our customer might have.

sgpennebaker

comment created time in 2 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

+//+//  File.m+//+//+//  Created by Dante Tam on 6/10/21.+//++#import <Foundation/Foundation.h>+#import <Amplitude.h>+#import "AMPStorage.h"++@implementation AMPStorage+++ (NSString *)getAppStorageAmpDir:(NSString *)instanceName {+    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSApplicationSupportDirectory, NSUserDomainMask, YES);+    NSString *path = [paths firstObject];+    NSString *bundleIdentifier = [[NSBundle mainBundle] bundleIdentifier];+    return [NSString stringWithFormat:@"%@/%@/%@", path, bundleIdentifier, instanceName];

I just tested it when I made the PR. It goes to $default_instance

dantetam

comment created time in 2 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

+//+//  File.m+//+//+//  Created by Dante Tam on 6/10/21.+//++#import <Foundation/Foundation.h>+#import <Amplitude.h>+#import "AMPStorage.h"++@implementation AMPStorage+++ (NSString *)getAppStorageAmpDir:(NSString *)instanceName {+    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSApplicationSupportDirectory, NSUserDomainMask, YES);+    NSString *path = [paths firstObject];+    NSString *bundleIdentifier = [[NSBundle mainBundle] bundleIdentifier];+    return [NSString stringWithFormat:@"%@/%@/%@", path, bundleIdentifier, instanceName];

What if instanceName is nil?

dantetam

comment created time in 2 days

push eventamplitude/Amplitude-iOS

Dante Tam

commit sha 0e2d08b778fe3766e07cdf69c32d37e5d43bcdab

reset some memory variables, add instance name as parameter to saved event files

view details

push time in 2 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

+//+//  File.m+//+//+//  Created by Dante Tam on 6/10/21.+//++#import <Foundation/Foundation.h>+#import <Amplitude.h>+#import "AMPStorage.h"++@implementation AMPStorage+++ (NSString *)getAppStorageAmpDir {+    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSApplicationSupportDirectory, NSUserDomainMask, YES);+    NSString *path = [paths firstObject];+    NSString *bundleIdentifier = [[NSBundle mainBundle] bundleIdentifier];+    return [NSString stringWithFormat:@"%@/%@/%@", path, @"/", bundleIdentifier];+}+++ (NSString *)getDefaultEventsFile {+    NSString *baseDir = [AMPStorage getAppStorageAmpDir];+    NSString *path = [baseDir stringByAppendingString:@"/amplitude_event_storage.txt"];

Oh yeah, if we have multiple instances, we should use different files. Instance name should be passed as a parameter now

dantetam

comment created time in 3 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

+//+//  File.m+//+//+//  Created by Dante Tam on 6/10/21.+//++#import <Foundation/Foundation.h>+#import <Amplitude.h>+#import "AMPStorage.h"++@implementation AMPStorage+++ (NSString *)getAppStorageAmpDir {+    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSApplicationSupportDirectory, NSUserDomainMask, YES);+    NSString *path = [paths firstObject];+    NSString *bundleIdentifier = [[NSBundle mainBundle] bundleIdentifier];+    return [NSString stringWithFormat:@"%@/%@/%@", path, @"/", bundleIdentifier];+}+++ (NSString *)getDefaultEventsFile {+    NSString *baseDir = [AMPStorage getAppStorageAmpDir];+    NSString *path = [baseDir stringByAppendingString:@"/amplitude_event_storage.txt"];

Another question here. Originally, we have different databases for different instance names. Should we also consider naming .txt file-related instance name here?

dantetam

comment created time in 3 days

issue commentamplitude/Amplitude-JavaScript

Is there any possibility the device id to be duplicated?

Hey @jason-hwang,

  1. We use v4 UUID generation for device_ids so the theoretical entropy of this is 1 collision in ~10^18 device ID's, in practice limited by the entropy of JS's Math.random(). My gut feeling is that 400 collisions is still a little high for 100,000 ID's so happy to work with you to debug this. How did you come about finding this issue?

  2. You can use that to regenerate the device ID, yes.

jason-hwang

comment created time in 3 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

 - (void)makeEventUploadPostRequest:(NSString *)url events:(NSString *)events num                 // If blocked by one massive event, drop it                 if (numEvents == 1) {                     if (maxEventId >= 0) {-                        (void) [self.dbHelper removeEvent:maxEventId];+                        self->_eventsBuffer = [[NSMutableArray alloc] init];

Also fixed this.

dantetam

comment created time in 3 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

 - (void)makeEventUploadPostRequest:(NSString *)url events:(NSString *)events num                     // success, remove existing events from dictionary                     uploadSuccessful = YES;                     if (maxEventId >= 0) {-                        (void) [self.dbHelper removeEvents:maxEventId];+                        [AMPStorage remove:[AMPStorage getDefaultEventsFile]];

Good catch.

dantetam

comment created time in 3 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

 - (void)makeEventUploadPostRequest:(NSString *)url events:(NSString *)events num                     // success, remove existing events from dictionary                     uploadSuccessful = YES;                     if (maxEventId >= 0) {-                        (void) [self.dbHelper removeEvents:maxEventId];+                        [AMPStorage remove:[AMPStorage getDefaultEventsFile]];

Do we need to handle the _eventsBuffer in this case?

dantetam

comment created time in 3 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

 - (void)makeEventUploadPostRequest:(NSString *)url events:(NSString *)events num                 // If blocked by one massive event, drop it                 if (numEvents == 1) {                     if (maxEventId >= 0) {-                        (void) [self.dbHelper removeEvent:maxEventId];+                        self->_eventsBuffer = [[NSMutableArray alloc] init];

If it falls into this case, do we need to anything about the file? I guess the massive event in this case is already saved on the file?

dantetam

comment created time in 3 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

 - (void)enterBackground {         self->_inForeground = NO;         [self refreshSessionTime:now];         [self uploadEventsWithLimit:0];+        [AMPStorage finish:[AMPStorage getDefaultEventsFile]];

That's a good catch. One simple way of handling that is to run finish if we see a JSON parsing error anywhere, and then try again

dantetam

comment created time in 3 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

 - (void)uploadEventsWithLimit:(int)limit {             return;         } -        long eventCount = [self.dbHelper getTotalEventCount];+        long eventCount = [self->_eventsBuffer count];         long numEvents = limit > 0 ? fminl(eventCount, limit) : eventCount;         if (numEvents == 0) {             self->_updatingCurrently = NO;             [self endBackgroundTaskIfNeeded];             return;         }-        NSMutableArray *events = [self.dbHelper getEvents:-1 limit:numEvents];-        NSMutableArray *identifys = [self.dbHelper getIdentifys:-1 limit:numEvents];-        NSDictionary *merged = [self mergeEventsAndIdentifys:events identifys:identifys numEvents:numEvents];+        +        long identifyCount = [self->_identifyBuffer count];+        long numIdentify = limit > 0 ? fminl(identifyCount, limit) : identifyCount;+        +        NSMutableArray *events = [[self->_eventsBuffer subarrayWithRange:NSMakeRange(0, numEvents)] mutableCopy];+        NSMutableArray *identifys = [[self->_identifyBuffer subarrayWithRange:NSMakeRange(0, numIdentify)] mutableCopy];+        NSDictionary *merged = [self mergeEventsAndIdentifys:events identifys:identifys numEvents:(numEvents+numIdentify)]; +        for (NSDictionary *event in events) {+            // convert event dictionary to JSON String+            NSError *error = nil;+            NSData *jsonData = [NSJSONSerialization dataWithJSONObject:[AMPUtils makeJSONSerializable:event] options:0 error:&error];+            if (error != nil) {+                AMPLITUDE_ERROR(@"ERROR: could not JSONSerialize event type: %@", error);+                continue;+            }+            NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];+            if ([AMPUtils isEmptyString:jsonString]) {+                AMPLITUDE_ERROR(@"ERROR: JSONSerializing event type event resulted in an NULL string");+                continue;+            }+            [AMPStorage storeEvent:jsonString];

Yes, there's better improvements for doing it as a batch. It's better to write to disk at once than many times elsewhere. Maybe this approach could go back to streaming if we used NSOutputStream. I got really good results using a batch, see the slack chat where I zipped some time profiles.

dantetam

comment created time in 3 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

 - (void)uploadEventsWithLimit:(int)limit {             return;         } -        long eventCount = [self.dbHelper getTotalEventCount];+        long eventCount = [self->_eventsBuffer count];         long numEvents = limit > 0 ? fminl(eventCount, limit) : eventCount;         if (numEvents == 0) {             self->_updatingCurrently = NO;             [self endBackgroundTaskIfNeeded];             return;         }-        NSMutableArray *events = [self.dbHelper getEvents:-1 limit:numEvents];-        NSMutableArray *identifys = [self.dbHelper getIdentifys:-1 limit:numEvents];-        NSDictionary *merged = [self mergeEventsAndIdentifys:events identifys:identifys numEvents:numEvents];+        +        long identifyCount = [self->_identifyBuffer count];+        long numIdentify = limit > 0 ? fminl(identifyCount, limit) : identifyCount;+        +        NSMutableArray *events = [[self->_eventsBuffer subarrayWithRange:NSMakeRange(0, numEvents)] mutableCopy];+        NSMutableArray *identifys = [[self->_identifyBuffer subarrayWithRange:NSMakeRange(0, numIdentify)] mutableCopy];+        NSDictionary *merged = [self mergeEventsAndIdentifys:events identifys:identifys numEvents:(numEvents+numIdentify)]; +        for (NSDictionary *event in events) {+            // convert event dictionary to JSON String+            NSError *error = nil;+            NSData *jsonData = [NSJSONSerialization dataWithJSONObject:[AMPUtils makeJSONSerializable:event] options:0 error:&error];+            if (error != nil) {+                AMPLITUDE_ERROR(@"ERROR: could not JSONSerialize event type: %@", error);+                continue;+            }+            NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];+            if ([AMPUtils isEmptyString:jsonString]) {+                AMPLITUDE_ERROR(@"ERROR: JSONSerializing event type event resulted in an NULL string");+                continue;+            }+            [AMPStorage storeEvent:jsonString];

So we are changing from storeEvent on event creation, and we save the events to file before upload happens. Is there any difference or performance improvements we can see by this way? And with this change, we just rely on memory more and can still lose events before we call upload?

dantetam

comment created time in 3 days

Pull request review commentamplitude/Amplitude-iOS

perf: Localstorage I/O optimization and batching strategies, streaming to disk

 - (void)enterBackground {         self->_inForeground = NO;         [self refreshSessionTime:now];         [self uploadEventsWithLimit:0];+        [AMPStorage finish:[AMPStorage getDefaultEventsFile]];

What would happen if this not get triggered? Like killed by memory management? That time this may not be triggered.

dantetam

comment created time in 3 days

push eventamplitude/Amplitude-iOS

Dante Tam

commit sha 113d4b9a700f5e77d78166692e4063bbafc04303

Remove from old dbhelper operations, fix PR more to align with my ticket

view details

push time in 3 days

issue commentamplitude/Amplitude-Java

SDK did not exception when wrong apikey, also do you have the way to enable log when using sdk ?

Hi @dantetam , Thank you for quickly response on this. I have just take a look at sdk source code. Could you please consider change apiKey parameter from String to char[] ? Cause I am using char[] to store key as below advice. I can change to String when call Amplitude but I think it's better if char[] is used. https://www.baeldung.com/java-storing-passwords

image

TuanDang1509

comment created time in 3 days

issue openedamplitude/Amplitude-JavaScript

Is there any possibility the device id to be duplicated?

Summary

The duplicated device id is observed in my service. (almost 100,000 device ids is collected)

Question 1

Is there any possibility the device id to be duplicated?

Question 2

If Q1 yes, to avoid the situation, does I have to use the re-generate method of amplitude's instance? -> amplitude.getInstance().regenerateDeviceId()

created time in 3 days

issue commentamplitude/Amplitude-Java

SDK did not exception when wrong apikey, also do you have the way to enable log when using sdk ?

Good catch on the Java version issue @TuanDang1509 . That's been fixed now.

That's also good reporting for the wrong API key case. We'll improve on that.

TuanDang1509

comment created time in 3 days