Global Game Jam 2020
Tools for Mystery Hunt
The official AWS X-Ray Recorder SDK for Java.
pull request commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
fix the error in build?
comment created time in 5 hours
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
+//+// Amplitude.m+// Copyright (c) 2013 Amplitude Inc. (https://amplitude.com/)
same question for the year here. And do we plan to add some test to cover this file?
comment created time in 5 hours
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
+//+// Amplitude.m+// Copyright (c) 2013 Amplitude Inc. (https://amplitude.com/)
Is this year should be up to date?
comment created time in 5 hours
push eventamplitude/Amplitude-ReactNative
commit sha 86036692b4d32d202585b3c028390400ddc81cc5
Added support for tvOS
push time in 12 hours
PR merged amplitude/Amplitude-ReactNative
pr closed time in 12 hours
issue closedamplitude/Amplitude-JavaScript
Is there any possibility the device id to be duplicated?
Summary
The duplicated device id is observed in my service.
- total: almost 100,000 device ids
- duplicated: almost 400 device ids
Question 1
Is there any possibility the device id to be duplicated?
Question 2
If Q1 yes, to avoid the situation, does I have to use the re-generate method of amplitude's instance?
-> amplitude.getInstance().regenerateDeviceId()
closed time in 19 hours
jason-hwang
issue commentamplitude/Amplitude-JavaScript
Is there any possibility the device id to be duplicated?
Thank you for the answer @kelvin-lu
Sorry for my mistake, I've realized there was a problem with my service logic. I didn't consider the users who are using multi-login in their one device. I will make a new issue If I have any questions.
Great thanks again 😄
comment created time in 19 hours
push eventamplitude/Amplitude-iOS
commit sha a52d34fb142d3b927ef4ef50cd5766ecdef8b096
Add copyright header for Amplitude
push time in 2 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
+//
Oh sorry, that's the wrong header. I didn't realize you were talking about a copyright
comment created time in 2 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
+//
Why is is resolved? Amplitude copyright is needed for all new files.
comment created time in 2 days
push eventamplitude/Amplitude-iOS
commit sha 3bffb3b26671dd16eccf804a1f773d0d8f29c9e6
Fix error case of unclosed JSON (no ending bracket), try finishing the file and do it again
push time in 2 days
issue commentamplitude/Amplitude-Java
Trying to create a simple Java example, part 2
Updates: We added the IMPORTANT NOTE to the developer center for this issue.
Thanks for letting us know the potential problem our customer might have.
comment created time in 2 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
+//+// File.m+//+//+// Created by Dante Tam on 6/10/21.+//++#import <Foundation/Foundation.h>+#import <Amplitude.h>+#import "AMPStorage.h"++@implementation AMPStorage+++ (NSString *)getAppStorageAmpDir:(NSString *)instanceName {+ NSArray *paths = NSSearchPathForDirectoriesInDomains(NSApplicationSupportDirectory, NSUserDomainMask, YES);+ NSString *path = [paths firstObject];+ NSString *bundleIdentifier = [[NSBundle mainBundle] bundleIdentifier];+ return [NSString stringWithFormat:@"%@/%@/%@", path, bundleIdentifier, instanceName];
I just tested it when I made the PR. It goes to $default_instance
comment created time in 2 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
+//+// File.m+//+//+// Created by Dante Tam on 6/10/21.+//++#import <Foundation/Foundation.h>+#import <Amplitude.h>+#import "AMPStorage.h"++@implementation AMPStorage+++ (NSString *)getAppStorageAmpDir:(NSString *)instanceName {+ NSArray *paths = NSSearchPathForDirectoriesInDomains(NSApplicationSupportDirectory, NSUserDomainMask, YES);+ NSString *path = [paths firstObject];+ NSString *bundleIdentifier = [[NSBundle mainBundle] bundleIdentifier];+ return [NSString stringWithFormat:@"%@/%@/%@", path, bundleIdentifier, instanceName];
What if instanceName is nil?
comment created time in 2 days
push eventamplitude/Amplitude-iOS
commit sha 0e2d08b778fe3766e07cdf69c32d37e5d43bcdab
reset some memory variables, add instance name as parameter to saved event files
push time in 2 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
+//+// File.m+//+//+// Created by Dante Tam on 6/10/21.+//++#import <Foundation/Foundation.h>+#import <Amplitude.h>+#import "AMPStorage.h"++@implementation AMPStorage+++ (NSString *)getAppStorageAmpDir {+ NSArray *paths = NSSearchPathForDirectoriesInDomains(NSApplicationSupportDirectory, NSUserDomainMask, YES);+ NSString *path = [paths firstObject];+ NSString *bundleIdentifier = [[NSBundle mainBundle] bundleIdentifier];+ return [NSString stringWithFormat:@"%@/%@/%@", path, @"/", bundleIdentifier];+}+++ (NSString *)getDefaultEventsFile {+ NSString *baseDir = [AMPStorage getAppStorageAmpDir];+ NSString *path = [baseDir stringByAppendingString:@"/amplitude_event_storage.txt"];
Oh yeah, if we have multiple instances, we should use different files. Instance name should be passed as a parameter now
comment created time in 3 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
+//+// File.m+//+//+// Created by Dante Tam on 6/10/21.+//++#import <Foundation/Foundation.h>+#import <Amplitude.h>+#import "AMPStorage.h"++@implementation AMPStorage+++ (NSString *)getAppStorageAmpDir {+ NSArray *paths = NSSearchPathForDirectoriesInDomains(NSApplicationSupportDirectory, NSUserDomainMask, YES);+ NSString *path = [paths firstObject];+ NSString *bundleIdentifier = [[NSBundle mainBundle] bundleIdentifier];+ return [NSString stringWithFormat:@"%@/%@/%@", path, @"/", bundleIdentifier];+}+++ (NSString *)getDefaultEventsFile {+ NSString *baseDir = [AMPStorage getAppStorageAmpDir];+ NSString *path = [baseDir stringByAppendingString:@"/amplitude_event_storage.txt"];
Another question here. Originally, we have different databases for different instance names. Should we also consider naming .txt file-related instance name here?
comment created time in 3 days
issue commentamplitude/Amplitude-JavaScript
Is there any possibility the device id to be duplicated?
Hey @jason-hwang,
-
We use v4 UUID generation for device_ids so the theoretical entropy of this is 1 collision in ~10^18 device ID's, in practice limited by the entropy of JS's
Math.random(). My gut feeling is that 400 collisions is still a little high for 100,000 ID's so happy to work with you to debug this. How did you come about finding this issue? -
You can use that to regenerate the device ID, yes.
comment created time in 3 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
- (void)makeEventUploadPostRequest:(NSString *)url events:(NSString *)events num // If blocked by one massive event, drop it if (numEvents == 1) { if (maxEventId >= 0) {- (void) [self.dbHelper removeEvent:maxEventId];+ self->_eventsBuffer = [[NSMutableArray alloc] init];
Also fixed this.
comment created time in 3 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
- (void)makeEventUploadPostRequest:(NSString *)url events:(NSString *)events num // success, remove existing events from dictionary uploadSuccessful = YES; if (maxEventId >= 0) {- (void) [self.dbHelper removeEvents:maxEventId];+ [AMPStorage remove:[AMPStorage getDefaultEventsFile]];
Good catch.
comment created time in 3 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
- (void)makeEventUploadPostRequest:(NSString *)url events:(NSString *)events num // success, remove existing events from dictionary uploadSuccessful = YES; if (maxEventId >= 0) {- (void) [self.dbHelper removeEvents:maxEventId];+ [AMPStorage remove:[AMPStorage getDefaultEventsFile]];
Do we need to handle the _eventsBuffer in this case?
comment created time in 3 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
- (void)makeEventUploadPostRequest:(NSString *)url events:(NSString *)events num // If blocked by one massive event, drop it if (numEvents == 1) { if (maxEventId >= 0) {- (void) [self.dbHelper removeEvent:maxEventId];+ self->_eventsBuffer = [[NSMutableArray alloc] init];
If it falls into this case, do we need to anything about the file? I guess the massive event in this case is already saved on the file?
comment created time in 3 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
- (void)enterBackground { self->_inForeground = NO; [self refreshSessionTime:now]; [self uploadEventsWithLimit:0];+ [AMPStorage finish:[AMPStorage getDefaultEventsFile]];
That's a good catch. One simple way of handling that is to run finish if we see a JSON parsing error anywhere, and then try again
comment created time in 3 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
- (void)uploadEventsWithLimit:(int)limit { return; } - long eventCount = [self.dbHelper getTotalEventCount];+ long eventCount = [self->_eventsBuffer count]; long numEvents = limit > 0 ? fminl(eventCount, limit) : eventCount; if (numEvents == 0) { self->_updatingCurrently = NO; [self endBackgroundTaskIfNeeded]; return; }- NSMutableArray *events = [self.dbHelper getEvents:-1 limit:numEvents];- NSMutableArray *identifys = [self.dbHelper getIdentifys:-1 limit:numEvents];- NSDictionary *merged = [self mergeEventsAndIdentifys:events identifys:identifys numEvents:numEvents];+ + long identifyCount = [self->_identifyBuffer count];+ long numIdentify = limit > 0 ? fminl(identifyCount, limit) : identifyCount;+ + NSMutableArray *events = [[self->_eventsBuffer subarrayWithRange:NSMakeRange(0, numEvents)] mutableCopy];+ NSMutableArray *identifys = [[self->_identifyBuffer subarrayWithRange:NSMakeRange(0, numIdentify)] mutableCopy];+ NSDictionary *merged = [self mergeEventsAndIdentifys:events identifys:identifys numEvents:(numEvents+numIdentify)]; + for (NSDictionary *event in events) {+ // convert event dictionary to JSON String+ NSError *error = nil;+ NSData *jsonData = [NSJSONSerialization dataWithJSONObject:[AMPUtils makeJSONSerializable:event] options:0 error:&error];+ if (error != nil) {+ AMPLITUDE_ERROR(@"ERROR: could not JSONSerialize event type: %@", error);+ continue;+ }+ NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];+ if ([AMPUtils isEmptyString:jsonString]) {+ AMPLITUDE_ERROR(@"ERROR: JSONSerializing event type event resulted in an NULL string");+ continue;+ }+ [AMPStorage storeEvent:jsonString];
Yes, there's better improvements for doing it as a batch. It's better to write to disk at once than many times elsewhere. Maybe this approach could go back to streaming if we used NSOutputStream. I got really good results using a batch, see the slack chat where I zipped some time profiles.
comment created time in 3 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
- (void)uploadEventsWithLimit:(int)limit { return; } - long eventCount = [self.dbHelper getTotalEventCount];+ long eventCount = [self->_eventsBuffer count]; long numEvents = limit > 0 ? fminl(eventCount, limit) : eventCount; if (numEvents == 0) { self->_updatingCurrently = NO; [self endBackgroundTaskIfNeeded]; return; }- NSMutableArray *events = [self.dbHelper getEvents:-1 limit:numEvents];- NSMutableArray *identifys = [self.dbHelper getIdentifys:-1 limit:numEvents];- NSDictionary *merged = [self mergeEventsAndIdentifys:events identifys:identifys numEvents:numEvents];+ + long identifyCount = [self->_identifyBuffer count];+ long numIdentify = limit > 0 ? fminl(identifyCount, limit) : identifyCount;+ + NSMutableArray *events = [[self->_eventsBuffer subarrayWithRange:NSMakeRange(0, numEvents)] mutableCopy];+ NSMutableArray *identifys = [[self->_identifyBuffer subarrayWithRange:NSMakeRange(0, numIdentify)] mutableCopy];+ NSDictionary *merged = [self mergeEventsAndIdentifys:events identifys:identifys numEvents:(numEvents+numIdentify)]; + for (NSDictionary *event in events) {+ // convert event dictionary to JSON String+ NSError *error = nil;+ NSData *jsonData = [NSJSONSerialization dataWithJSONObject:[AMPUtils makeJSONSerializable:event] options:0 error:&error];+ if (error != nil) {+ AMPLITUDE_ERROR(@"ERROR: could not JSONSerialize event type: %@", error);+ continue;+ }+ NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];+ if ([AMPUtils isEmptyString:jsonString]) {+ AMPLITUDE_ERROR(@"ERROR: JSONSerializing event type event resulted in an NULL string");+ continue;+ }+ [AMPStorage storeEvent:jsonString];
So we are changing from storeEvent on event creation, and we save the events to file before upload happens. Is there any difference or performance improvements we can see by this way? And with this change, we just rely on memory more and can still lose events before we call upload?
comment created time in 3 days
Pull request review commentamplitude/Amplitude-iOS
perf: Localstorage I/O optimization and batching strategies, streaming to disk
- (void)enterBackground { self->_inForeground = NO; [self refreshSessionTime:now]; [self uploadEventsWithLimit:0];+ [AMPStorage finish:[AMPStorage getDefaultEventsFile]];
What would happen if this not get triggered? Like killed by memory management? That time this may not be triggered.
comment created time in 3 days
push eventamplitude/Amplitude-iOS
commit sha 113d4b9a700f5e77d78166692e4063bbafc04303
Remove from old dbhelper operations, fix PR more to align with my ticket
push time in 3 days
issue commentamplitude/Amplitude-Java
SDK did not exception when wrong apikey, also do you have the way to enable log when using sdk ?
Hi @dantetam , Thank you for quickly response on this. I have just take a look at sdk source code. Could you please consider change apiKey parameter from String to char[] ? Cause I am using char[] to store key as below advice. I can change to String when call Amplitude but I think it's better if char[] is used. https://www.baeldung.com/java-storing-passwords
comment created time in 3 days
issue openedamplitude/Amplitude-JavaScript
Is there any possibility the device id to be duplicated?
Summary
The duplicated device id is observed in my service. (almost 100,000 device ids is collected)
Question 1
Is there any possibility the device id to be duplicated?
Question 2
If Q1 yes, to avoid the situation, does I have to use the re-generate method of amplitude's instance?
-> amplitude.getInstance().regenerateDeviceId()
created time in 3 days
issue commentamplitude/Amplitude-Java
SDK did not exception when wrong apikey, also do you have the way to enable log when using sdk ?
Good catch on the Java version issue @TuanDang1509 . That's been fixed now.
That's also good reporting for the wrong API key case. We'll improve on that.
comment created time in 3 days