My blog
Laravel echo app
Vuejs and SEO with Laravel
Slam Dunk JavaScript
Bulma webpack
My dotfiles
Vanilla JavaScript emoji picker component
F1 races by timezone
pull request commentmatomo-org/matomo
Make archiving process to respect process_new_segments_from settings
@flamisz @tsteur left two new comments
comment created time in 42 minutes
issue commentmatomo-org/matomo
When requesting Date Range or Custom Segment, only archive the requested record
I was more meaning only adding a major issue but not scheduling it yet but added it for now to 4.5 milestone as it would be good to have but not 100% sure on it's impact yet (in some cases it be quite a big impact though). To be seen how easy it is to develop.
comment created time in an hour
Pull request review commentmatomo-org/matomo
Make archiving process to respect process_new_segments_from settings
public function canSkipThisArchive() $hasSiteVisitsBetweenTimeframe = $this->hasSiteVisitsBetweenTimeframe($idSite, $params->getPeriod()); $hasChildArchivesInPeriod = $this->dataAccessModel->hasChildArchivesInPeriod($idSite, $params->getPeriod()); + if ($this->canSkipArchiveForSegment()) {+ return true;+ }+ return $isWebsiteUsingTracker && !$isArchivingForcedWhenNoVisits && !$hasSiteVisitsBetweenTimeframe && !$hasChildArchivesInPeriod; } + public function canSkipArchiveForSegment()+ {+ $params = $this->params;++ if ($params->getSegment()->isEmpty()) {+ return false;+ }++ $idSite = $params->getSite()->getId();+ $periodEnd = $params->getPeriod()->getDateEnd();+ $segmentHash = $params->getSegment()->getHash();+ /** @var SegmentArchiving */+ $segmentArchiving = StaticContainer::get(SegmentArchiving::class);+ $segmentInfo = $segmentArchiving->findSegmentForHash($segmentHash, $idSite);++ if (!$segmentInfo) {+ return false;+ }++ $segmentArchiveStartDate = $segmentArchiving->getReArchiveSegmentStartDate($segmentInfo);++ if ($segmentArchiveStartDate !==null && $segmentArchiveStartDate->isLater($periodEnd->getEndOfDay())) {+ $doneFlag = Rules::getDoneStringFlagFor(+ [$idSite],+ $params->getSegment(),+ $params->getPeriod()->getLabel(),+ $params->getRequestedPlugin()+ );++ // no specific report request+ // if there is no invalidation where the report is null, we can skip+ // if we have invalidations for the period and name, but only for a specific reports, we can skip+ if (!$params->getArchiveOnlyReport() && !$this->dataAccessModel->hasInvalidationForPeriodAndName($idSite, $params->getPeriod(), $doneFlag)) {+ return true;+ }++ // specific report request+ // we only want to rearchive if we have invalidation for that report+ // if we don't find invalidation for that report, we can skip+ if ($params->getArchiveOnlyReport() && !$this->dataAccessModel->hasInvalidationForPeriodAndName($idSite, $params->getPeriod(), $doneFlag, $params->getArchiveOnlyReport())) {+ return true;+ }
Could the two ifs above be simplified to just:
return !$this->dataAccessModel->hasInvalidationForPeriodAndName($idSite, $params->getPeriod(), $doneFlag, $params->getArchiveOnlyReport());
?
Since archive only report will be empty when it's not set, we can just pass it as is, right?
comment created time in an hour
Pull request review commentmatomo-org/matomo
Make archiving process to respect process_new_segments_from settings
private function invalidateWithSegments($idSites, $date, $period, $_forceInvalid if ($this->canWeSkipInvalidatingBecauseThereIsAUsablePeriod($params, $doNotIncludeTtlInExistingArchiveCheck)) { $this->logger->debug(' Found usable archive for {archive}, skipping invalidation.', ['archive' => $params]); } else {+ if (empty($this->segmentArchiving)) {+ // might not be initialised if init is not called+ $this->segmentArchiving = StaticContainer::get(SegmentArchiving::class);+ }+ $segmentHash = $params->getSegment()->getHash();+ $segmentInfo = $this->segmentArchiving->findSegmentForHash($segmentHash, $idSite);+ $periodEnd = $params->getPeriod()->getDateEnd();++ if ($segmentInfo) {+ $segmentArchiveStartDate = $this->segmentArchiving->getReArchiveSegmentStartDate($segmentInfo);++ if ($segmentArchiveStartDate !== null && $segmentArchiveStartDate->isLater($periodEnd->getEndOfDay())) {+ // the system is not allowed to invalidate reports for this period+ // automatically, only a user can specifically invalidate+ continue;+ }+ }
@flamisz this wasn't addressed, did you discuss it w/ @tsteur or something?
comment created time in an hour
issue commentmatomo-org/matomo
When requesting Date Range or Custom Segment, only archive the requested record
@tsteur this would be a high value improvement for UI performance and reducing DB usage, significantly in some cases. Sounds good to schedule it in a 4.x.0 :rocket:
comment created time in an hour
push eventmatomo-org/developer-documentation
commit sha b5f46f9d494b699356c0c5a34261704a0ca9e80d
updated plugins API documentation
push time in 3 hours
delete branch matomo-org/matomo
delete branch : allow-invalidating-empty-site
delete time in 3 hours
push eventmatomo-org/matomo
commit sha 929d2ffe7d3392ece2348a50315576f349668b53
Allow invalidating empty site (#17519) * allow invalidating empty sites (since some sites can have no visits but need to be invalidated at week or higher periods) * if skipping archiving, and there are existing archives found, use them instead of assuming 0 visits * fix test
push time in 3 hours
PR merged matomo-org/matomo
Description:
Some sites can legitimately have 0 visits, like sites imported from google. While debugging the GoogleAnalyticsImporter test failures I noticed nothing was being invalidated because of this check. Removing it in this PR.
Review
- [ ] Functional review done
- [ ] Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
- [ ] Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
- [ ] Security review done see checklist
- [ ] Code review done
- [ ] Tests were added if useful/possible
- [ ] Reviewed for breaking changes
- [ ] Developer changelog updated if needed
- [ ] Documentation added if needed
- [ ] Existing documentation updated if needed
pr closed time in 3 hours
push eventmatomo-org/matomo
commit sha 538a1425d5e8cd9e449923a141d37741ea2f6582
fix test
push time in 4 hours
issue commentmatomo-org/matomo
Filtered monthly page views do not sum up to filtered yearly page views
@heurteph-ei there should be already a feature request for this 👍
comment created time in 5 hours
pull request commentmatomo-org/matomo
@tsteur found another regression causing the importer tests to fail, added in the latest commit.
comment created time in 6 hours
issue closedmatomo-org/matomo
Now that custom dimensions is included in core I think we could improve the performance of the tracker and have less updates.
Currently, as soon as a custom dimension is configured for a site, we execute an additional query like UPDATE log_visit SET last_idlink_va = '2222' WHERE idsite = '1' AND idvisit = '3333'.
We could maybe get rid of this extra update query for existing visits if we knew we first insert the action and then update the visit. Then we could in the regular visit update also update the last_idlink_va. Not sure if there would be maybe a reason this wouldn't work and whether we have to update the visit before the action.
For new visits this would not work because we need to have the idvisit to insert the action. This likely makes this quite tricky to implement...
I was thinking we could maybe also get rid of the extra action update but this is not the case because it actually updates the action before this action.
Note: the query is only executed when there are action custom dimensions defined so its impact is limited.
closed time in 6 hours
tsteur
push eventmatomo-org/matomo
commit sha c2f2ea56586e4c9c45232617329bee40898a19d0
if skipping archiving, and there are existing archives found, use them instead of assuming 0 visits
push time in 6 hours
push eventmatomo-org/matomo
commit sha bdd1172e6f853b227b208ab9daa67abe20435e66
Update RequestProcessor.php
push time in 6 hours
push eventmatomo-org/matomo
commit sha 9f743ba05532ab8870118d298e4191a482a8cad1
Update ActionDimension.php
push time in 6 hours
push eventmatomo-org/matomo
commit sha f113d2eac96e46538aee0832d9de221ff2187e5d
Update ActionDimension.php
push time in 6 hours
push eventmatomo-org/matomo
commit sha c26d59daa14e6aa71ea741437363d1c636c3fcaa
Update ActionDimension.php
push time in 6 hours
Pull request review commentmatomo-org/matomo
public function handle() $this->request->setThirdPartyCookie($this->request->getVisitorIdForThirdPartyCookie()); foreach ($this->requestProcessors as $processor) {+ if (!$isNewVisit && $processor instanceof ActionsRequestProcessor) {+ // already processed earlier when handling exisitng visit see {@link self::handleExistingVisit()}+ continue;+ }
I know it's not really clear which is why I'm not sure yet if it's worth merging it. Problem is for new visits we have to execute the actions processor later vs for existing visits we have to execute it before updating a visit...
comment created time in 7 hours
Pull request review commentmatomo-org/matomo
public function handle() $this->request->setThirdPartyCookie($this->request->getVisitorIdForThirdPartyCookie()); foreach ($this->requestProcessors as $processor) {+ if (!$isNewVisit && $processor instanceof ActionsRequestProcessor) {+ // already processed earlier when handling exisitng visit see {@link self::handleExistingVisit()}+ continue;+ }
oh i see, this has to be done before updating a visit... seems like it would all be clearer if this was all in ActionsRequestProcessor, though i'm not sure if that's possible
comment created time in 7 hours
Pull request review commentmatomo-org/matomo
public function handle() $this->request->setThirdPartyCookie($this->request->getVisitorIdForThirdPartyCookie()); foreach ($this->requestProcessors as $processor) {+ if (!$isNewVisit && $processor instanceof ActionsRequestProcessor) {+ // already processed earlier when handling exisitng visit see {@link self::handleExistingVisit()}+ continue;+ }
could this be put in ActionsRequestProcessor? ie, return if conditions are met?
comment created time in 7 hours
push eventmatomo-org/matomo
commit sha fd565cf3b49150458dbaf5610963adf1c2e4999e
more docs
push time in 7 hours
PR opened matomo-org/matomo
Description:
Performance improvement to have one less update on log_visit for existing visits if action dimensions are used: https://github.com/matomo-org/matomo/issues/17173
Review
- [ ] Functional review done
- [ ] Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
- [ ] Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
- [ ] Security review done see checklist
- [ ] Code review done
- [ ] Tests were added if useful/possible
- [ ] Reviewed for breaking changes
- [ ] Developer changelog updated if needed
- [ ] Documentation added if needed
- [ ] Existing documentation updated if needed
pr created time in 7 hours
push eventmatomo-org/matomo
commit sha a348ef67f1a8ec0cc38bf672cbbffad42c268fb6
add more comments
push time in 7 hours
pull request commentmatomo-org/matomo
@flamisz I think we can undo the changes in Access.php and apply this change:
--- a/core/FrontController.php
+++ b/core/FrontController.php
@@ -675,7 +675,8 @@ class FrontController extends Singleton
return null;
}
- if (Common::getRequestVar('token_auth', '', 'string') !== '') {
+ if (Common::getRequestVar('token_auth', '', 'string') !== ''
+ && !Common::getRequestVar('force_api_session')) {
return null;
}
It basically means if there's a force_api_session=1 parameter then we still load the session and authenticate the session first which is needed for the session token to work. This way the dashboard etc will work and all other requests should be working too. It be only an issue if users falsely have force_api_session=1 set in the request and don't use a session token but a real token auth. Then it won't work but then it generally shouldn't work anyway AFAIK.
It be still good for someone to review this though additionally.
comment created time in 7 hours
push eventmatomo-org/matomo
commit sha 879e8d528b74a057fcdcda4b6b2a9690bfcdc02b
fix tests
push time in 8 hours
push eventmatomo-org/matomo
commit sha 6af6934ddddce1778aa913c4aa7de47db97dddd8
try fix build
push time in 8 hours
push eventmatomo-org/matomo
commit sha 932377ea5b8a49af89c5f3cf0ced536c059048b7
try one less update
push time in 8 hours
push eventmatomo-org/matomo
commit sha a41eeda3bd9f64f509462714a0f0397dbad36902
update column
push time in 9 hours