mattab ( Matthieu Aubry )

commit sha e0108cae706de9aa7b4da9f1f9f8b2e80d4c0a9c

language update (#17648) Co-authored-by: tsteur <[email protected]>

push time in 26 minutes

delete branch matomo-org/matomo

delete branch : translationupdates

delete time in 26 minutes

PR merged matomo-org/matomo

[automatic translation update] Updated 26 strings in 5 languages (tr it el sq es) c: i18n

Updated Turkish (2 changes / 100% translated)
Updated Italian (18 changes / 99% translated)
Updated Greek (2 changes / 100% translated)
Updated Albanian (2 changes / 99% translated)
Updated Spanish (2 changes / 91% translated)

Help us translate Matomo in your language! Signup at https://www.transifex.com/matomo/matomo/ If you have any questions, get in touch with us at [email protected]

+26 -4

1 commit in matomo-org/matomo/tree/translationupdates matomo-org/matomo/tree/4.x-dev

0 comment

5 changed files

github-actions[bot]

pr closed time in 26 minutes

PR opened matomo-org/matomo

if cdt is not used during tracking, sort by idvisit descending when looking for existing visits

Description:

Refs #16904

Sorting by visit_last_action_time when looking for the most recent visit of a visitor can be slow, but sorting by idvisit desc is demonstrably fast. The only problem is, it can only be done when we know that visit rows in the table are in order. That is to say, that the visit_log_action_time values always increase as idvisit increases.

If a user tracks something in the past via the cdt parameter, this no longer becomes the case and we can't perform this optimization. This PR sets an option when cdt is used and, if the option is NOT set, uses the idvisit DESC optimization.

There is also a command that can be used to detect if log_visit is out of order and set the option. It might take a long time to run those queries, so we don't do it automatically or in the included update.

The original problem will still exist for sites where cdt was used.

Review

[ ] Functional review done
[ ] Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
[ ] Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
[ ] Security review done see checklist
[ ] Code review done
[ ] Tests were added if useful/possible
[ ] Reviewed for breaking changes
[ ] Developer changelog updated if needed
[ ] Documentation added if needed
[ ] Existing documentation updated if needed

+182 -13

5 commit in matomo-org/matomo/tree/idvisit-desc-optimization matomo-org/matomo/tree/4.x-dev

0 comment

8 changed files

pr created time in 5 hours

push eventmatomo-org/matomo

commit sha c3d7ba58555d87a4dcaa32f88f4fa6fac55f7170

output more information about out of order visits so the result can be verified if needed

push time in 5 hours

push eventmatomo-org/matomo

commit sha 8b4b8ce017c15a534101fc10494a9f05cb824a58

turn on github actions (#17548)

Lukas Winkler

commit sha 72a83dedec5d1c86237f6b5a5857f9b448662b8b

fix usort() in SQL Profiler (#17645)

Lukas Winkler

commit sha 6b091d2f117b14ecc08fcb05b39ed654df4042a8

use strict-origin-when-cross-origin Referrer-Policy (#17382)

commit sha 0cbacd31875aba3678de3eb4bbbf3297ec9453ba

updates all submodules (#17644) Co-authored-by: diosmosis <[email protected]>

commit sha 1a6662bbb1502dbaf6194016d509274850b74dfe

Merge branch '4.x-dev' into idvisit-desc-optimization

commit sha 780536171f9f80d79975cdc9b9865c0b1afa9850

Fix option prefixing + add new update to assume cdt has been used

commit sha 78518602347c9cab78e820b8d3f2a06f9a6104ee

Add command to check if visit data is out of order.

branch : translationupdates

push time in 5 hours

create barnchmatomo-org/matomo

created branch time in 5 hours

PR opened matomo-org/matomo

[automatic translation update] Updated 26 strings in 5 languages (tr it el sq es)

Updated Turkish (2 changes / 100% translated)
Updated Italian (18 changes / 99% translated)
Updated Greek (2 changes / 100% translated)
Updated Albanian (2 changes / 99% translated)
Updated Spanish (2 changes / 91% translated)

Help us translate Matomo in your language! Signup at https://www.transifex.com/matomo/matomo/ If you have any questions, get in touch with us at [email protected]

+26 -4

1 commit in matomo-org/matomo/tree/translationupdates matomo-org/matomo/tree/4.x-dev

0 comment

5 changed files

pr created time in 5 hours

pull request commentmatomo-org/matomo

Add link to new FAQ how to make the diagnostic “Managing processes via CLI” show Ok

If we escape the comment we won't be able to put the link in, correct? We could accomplish the same thing w/ another variable that's meant to not contain unsafe input (eg, item.details after item.comment).

mattab

comment created time in 7 hours

push eventmatomo-org/matomo

commit sha 06b70bca904f2858dc14fd21fd0438505ea8c1cd

Move context addition to ExceptionToTextProcessor.

Performance metrics report incorrect DOM Processing & Completion times

push time in 7 hours

issue commentmatomo-org/matomo

Hi @MaghSamana, if you were to perform the same actions as this user, would you see the same performance metrics show up in Matomo (in other words, is the problem reliably reproducible)? Can you see what browser, os, internet provider the specific user is using?

MaghSamana

comment created time in 8 hours

push eventmatomo-org/matomo

commit sha f1b82cd9a2ed319676e0c505712055a5c310909c

use rawSafeDecoded filter (#17473)

commit sha 49f0499b480f96cd5e3a9e2abfd286c3c2d1d690

Update bug_report.md (#17477)

commit sha 647ed543aecd9b1d92ecfb95acd5f9ac1c7d58f7

Remove urlencode for segment definitions (#17475) * remove urlencode for segment definitions * use hash from db in model

commit sha b66635e4e16f5af26171211b6196f1dff319e1c5

show period label on evo graph (#17456) * v1 * v2 * use css classes * update period name when selected * fix ui tests * use translation for period name Co-authored-by: sgiehl <[email protected]>

commit sha 6d726c8005d6dd537ca7112b22d12ad70156af74

Use full segment data to rearchive segment (#17478) * Use full segment data to rearchive segment * fix test Co-authored-by: diosmosis <[email protected]>

commit sha 59bc5384af96672b3740d18a439295bc9948dc54

Do not use DbHelper::getTablesInstalled() in integration test setup since it posts event that loads all activated plugins. (#17480)

commit sha 29560389f7ba80d8d6d83ee4579d6eba2519e911

Do not parse plugin details with angular in marketplace overlay (#17483)

commit sha 3cee577117ad92db739dd3f22571520207eca02e

updates all submodules (#17489) Co-authored-by: diosmosis <[email protected]>

commit sha c29ee95a443f5e19ddf37781f0a3df8cc87ddfa6

Improve method to determine date to archive segments from (#17492) * Improve method to determine date to archive segments from * fix/add tests

commit sha faceca4eb3e17e769286ca5e437da8521192ca66

Fix OneClickUpdate UI test (#17494)

commit sha d1422903bb698fac80e1ab0590d62c63dd6574bf

Updates library pear/archive_tar (#17491)

commit sha 6227cb05197d4dfd0aa0d695eb665b6c1ef455d6

Require password confirmation for more plugin operations. (#17345) * Require password confirmation for more plugin operations. * renormalize * add optional password confirmation to CorePluginsAdmin.setSystemSettings * Add developer changelog entry. * ask for password confirmation when saving plugin settings and use onOpenEnd materializecss modal event handler instead of ready since ready no longer exists in used version * Fix redirectTo==referrer for other plugin actions that now have password confirmation. * fix build * try fixing build again

commit sha 88800290a3f997b84101bd874b143e6c5021261f

Add new update to make sure deleted segments also have a computed hash. (#17485) * Add new update to make sure deleted segments also have a computed hash. * Update 4.3.0-b4.php

commit sha fb9e10ff80bacea2643a5100bc7a85f2903c10bb

language update (#17495) Co-authored-by: tsteur <[email protected]>

commit sha 634a9a5176c54ed35db9e816e9b8f8569869ad01

Add diagnostic that checks if server directories that should be private are accessible (#17490) * add initial diagnostic * unfinished * Complete diagnostic and get to work. * Update plugins/Diagnostics/Diagnostic/RequiredPrivateDirectories.php Co-authored-by: Stefan Giehl <[email protected]> * apply review feedback * fixing some UI tests Co-authored-by: Stefan Giehl <[email protected]>

commit sha 88faa0add94dca67315d4cee31a451ef7cd3289b

work around group_concat max len being too small edge case for the select in ArchiveSelector.getArchiveIds (#17496)

Thomas Steur

commit sha ea8d3ca93cd499e8910fa4667f8b9fa4988e9993

Link to the Matomo font guide (#17504) Now that we have published a detailed guide for this let's link to the guide instead of the partial instructions.

commit sha dba04710b179e48f5a7ae5d967a849ad57b3bc22

updates all submodules (#17513) Co-authored-by: diosmosis <[email protected]>

commit sha 8e2531cb6cfb00fdbc88a4622398b162b91715c2

Fix determine segment rearchive date (#17505)

commit sha 2670f6508151c52ced827f1c3bde9e16f52af353

language update (#17514) Co-authored-by: tsteur <[email protected]>

The same access session produces two different visitorids

push time in 8 hours

issue commentmatomo-org/matomo

Hi @Amber-Wish, by default Matomo will look for an existing visit by visitor ID, and if none is found, try to look for one using the device fingerprint (within the last 30 mins), in case the visitor ID changes for some reason. If the config_id for the two visitor IDs is the same, this is what happened. If you don't want this to be done, you can disable this behavior: https://matomo.org/faq/how-to/faq_175/

Does this answer your question?

Amber-Wish

comment created time in 8 hours

issue openedmatomo-org/matomo

The same access session produces two different visitorids

hello, When I checked the data, I found the data in the following picture. Is this data normal? Why is this kind of data produced? I hope to get a reply. thank you.

Our Environment Matomo version: 4.3.1 PHP version: 7.4.19

created time in 15 hours

issue commentmatomo-org/matomo

Goal report take ages to generate after an update

Should be fixed by the PRs in 4.3, closing this one.

nelhefni

comment created time in 15 hours

issue closedmatomo-org/matomo

Goal report take ages to generate after an update

Hello,

After updating an existing Goal, the next cron job will try to generate past reports.

My current settings: rearchive_reports_in_past_last_n_months = last1

The cron job will then take up to 30 hours to complete! Each weekly report took around 10 minutes. But it seems that the monthly report for February took 27 hours. Thousands of partial reports will be inserted in the DB during the process

MariaDB [matomo]> select count(*) from matomo_archive_numeric_2021_03 where date1 = '2021-03-22' and period = 1 and name like '%Goal%';
+----------+
| count(*) |
+----------+
|   125936 |

We have 125936 Goal reports for a single day. Filtering on the row including done and Goal:

MariaDB [matomo]> select count(*) from matomo_archive_numeric_2021_03 where date1 = '2021-03-22' and period = 1 and name like 'done%Goal%';
+----------+
| count(*) |
+----------+
|     3572 |
+----------+

Here is a small extract of the actual rows:

...
|    260168 | done5aa0e3c4a2c285b80bcc0cd6e8a62f3e.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:28:35 |     5 |
|    260169 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:28:35 |     5 |
|    260194 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:29:16 |     5 |
|    260196 | done5aa0e3c4a2c285b80bcc0cd6e8a62f3e.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:29:17 |     5 |
|    260197 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:29:17 |     5 |
|    260222 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:29:58 |     5 |
|    260224 | done5aa0e3c4a2c285b80bcc0cd6e8a62f3e.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:29:59 |     5 |
|    260225 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:30:00 |     5 |
|    260250 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:30:40 |     5 |
|    260252 | done5aa0e3c4a2c285b80bcc0cd6e8a62f3e.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:30:42 |     5 |
|    260253 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:30:42 |     5 |
|    260278 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:31:23 |     5 |
|    260280 | done5aa0e3c4a2c285b80bcc0cd6e8a62f3e.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:31:24 |     5 |
|    260281 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:31:25 |     5 |
|    260306 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:32:05 |     5 |
|    260308 | done5aa0e3c4a2c285b80bcc0cd6e8a62f3e.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:32:07 |     5 |
|    260309 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:32:08 |     5 |
|    260334 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:32:49 |     5 |
|    260336 | done5aa0e3c4a2c285b80bcc0cd6e8a62f3e.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:32:50 |     5 |
|    260337 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:32:51 |     5 |
|    260362 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:33:32 |     5 |
|    260364 | done5aa0e3c4a2c285b80bcc0cd6e8a62f3e.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:33:33 |     5 |
|    260365 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:33:34 |     5 |
|    260390 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:34:15 |     5 |
|    260392 | done5aa0e3c4a2c285b80bcc0cd6e8a62f3e.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:34:16 |     5 |
|    260393 | donefea44bece172bc9696ae57c26888bf8a.Goals |      1 | 2021-03-22 | 2021-03-22 |      1 | 2021-04-05 20:34:17 |     5 |
...

Goal plugin seems to be archiving segments for the same dates over and over. Could you please have a look?

Your Environment

Matomo 4.2.1

closed time in 15 hours

nelhefni

push eventmatomo-org/developer-documentation

Piwik Docs

commit sha 50c4cbb2ffc715c619e73aaff65d30870a580e6a

updated plugins API documentation

Regression: some daily reports disappear from UI, and weekly reports are corrupted

push time in 15 hours

issue closedmatomo-org/matomo

Hello,

We think PR https://github.com/matomo-org/matomo/pull/17216 has introduced some regressions to the archiving process since Matomo 4.2.0

FYI, browser archiving is disabled:

enable_browser_archiving_triggering = 0
browser_archiving_disabled_enforce = 1
archiving_range_force_on_browser_request = 0

For a given plugin, we are facing 2 issues since the update to Matomo 4.2.0,

Issue#1: Some daily reports disappear from the UI, even though delete_reports_enable is set to 0
Issue#2: Some weekly reports have inaccurate data: gather the values of only some days of the week.

This seems to be related to the changes made with the handling of partial archives. Indeed, we have a lot of partial archives in our DB, which doesn't contain the reports of our impacted plugins:

MariaDB [matomo]> select * from matomo_archive_numeric_2021_03 where idarchive = 46176;

| idarchive | name        | idsite   | date1      | date2      | period    | ts_archived         | value    |

|     46176 | myPlugin    |        1 | 2021-03-22 | 2021-03-22 |         1 | 2021-03-23 00:05:52 |       10 |
|     46176 | otherPlugin |        1 | 2021-03-22 | 2021-03-22 |         1 | 2021-03-23 00:05:52 |       20 |
|     46176 | done        |        1 | 2021-03-22 | 2021-03-22 |         1 | 2021-03-23 00:05:52 |        1 |

MariaDB [matomo]> select * from matomo_archive_numeric_2021_03 where idarchive = 56405;

| idarchive | name        | idsite   | date1      | date2      | period    | ts_archived         | value    |

|     56405 | otherPlugin |        1 | 2021-03-22 | 2021-03-22 |         1 | 2021-04-05 18:52:03 |       20 |
|     56405 | done        |        1 | 2021-03-22 | 2021-03-22 |         1 | 2021-04-05 18:52:03 |        5 |

As you can see, myPlugin report is not part of partial archive 56405 (partial because the value of done row is 5), but it is nevertheless selected since Matomo 4.2.0, as ts_archived is more recent.

Since https://github.com/matomo-org/matomo/pull/17216, checkAuthorizedToArchive = false is passed to Rules::getSelectableDoneFlagValues during the retrival of archive to display in the UI https://github.com/matomo-org/matomo/blob/2c30dc8d231af58cdf2516a413da2d86bf7d62c6/core/DataAccess/ArchiveSelector.php#L370

As a consequence, both ArchiveWriter::DONE_PARTIAL and ArchiveWriter::DONE_INVALIDATED are added to the list of accepted value. Which explains why archive 56405 is selected instead of 46176 in our case. https://github.com/matomo-org/matomo/blob/2c30dc8d231af58cdf2516a413da2d86bf7d62c6/core/ArchiveProcessor/Rules.php#L321

Thus, as soon as archive 56405 was created, myPlugin report disappeared from the UI. I don't think ArchiveWriter::DONE_PARTIAL should be part of the list of accepted values. We have been able to fix this issue#1 with this PR https://github.com/matomo-org/matomo/pull/17425

Concerning issue#2, we have been able to fix it by reverting the changes done in the same PR to ArchiveSelector::findArchiveDataWithLatestTsArchived, and then invalidating faulty reports and re-running the cron job.

Your Environment

Matomo 4.2.1

closed time in 15 hours

nelhefni

issue commentmatomo-org/matomo

Regression: some daily reports disappear from UI, and weekly reports are corrupted

Assuming this was all fixed in 4.3 as we're no issues so far. I'll close this.

nelhefni

comment created time in 15 hours

PR closed matomo-org/matomo

Fix issue with reports disappearing from UI Stale for long

Description:

Experimental fix for https://github.com/matomo-org/matomo/issues/17428 Some daily reports disappear from the UI, even though delete_reports_enable is set to 0

Review

[ ] Functional review done
[ ] Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
[ ] Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
[ ] Security review done see checklist
[ ] Code review done
[ ] Tests were added if useful/possible
[ ] Reviewed for breaking changes
[ ] Developer changelog updated if needed
[ ] Documentation added if needed
[ ] Existing documentation updated if needed

+4 -3

2 commit in nelhefni/matomo/tree/fix-issue-with-reports-disappearing-from-ui matomo-org/matomo/tree/4.x-dev

4 comments

1 changed file

nelhefni

pr closed time in 15 hours

pull request commentmatomo-org/matomo

Fix issue with reports disappearing from UI

The several PRs created for the connected issue should have solved this in 4.3, so going to close this one.

nelhefni

comment created time in 15 hours

delete branch matomo-org/matomo

delete branch : submodules

delete time in 15 hours

push eventmatomo-org/matomo

commit sha 0cbacd31875aba3678de3eb4bbbf3297ec9453ba

updates all submodules (#17644) Co-authored-by: diosmosis <[email protected]>

push time in 15 hours

PR merged matomo-org/matomo

[automatic submodule updates]

Updated submodules:

plugins/CustomAlerts

+1 -1

1 commit in matomo-org/matomo/tree/submodules matomo-org/matomo/tree/4.x-dev

0 comment

1 changed file

github-actions[bot]

pr closed time in 15 hours

delete branch matomo-org/matomo

delete branch : strict-origin-when-cross-origin

delete time in 15 hours

push eventmatomo-org/matomo

Lukas Winkler

commit sha 6b091d2f117b14ecc08fcb05b39ed654df4042a8

use strict-origin-when-cross-origin Referrer-Policy (#17382)