Commit Graph

105 Commits

Author SHA1 Message Date
James Teow
b70500ad86 Bug 1892267 - Add ads hidden and ads loaded to the categorization event - r=scunnane
This augments the existing metric that contains the number of ads that were visible at the
time the page was snapshot.

Differential Revision: https://phabricator.services.mozilla.com/D207935
2024-04-19 00:11:56 +00:00
Stephanie Cunnane
924cbf7d28 Bug 1877494 - Collect telemetry on Google account signed-in status. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D207206
2024-04-15 19:30:24 +00:00
James Teow
8c082e0c2c Bug 1889677 - Modify SERP categorization tests to wait for events - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D206909
2024-04-08 19:24:50 +00:00
James Teow
0ff995ac8c Bug 1889940 - Add additional logging for domain extraction - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D206777
2024-04-05 15:32:52 +00:00
James Teow
4935a2a56a Bug 1886095 - Store hashed domain to categories inside Sqlite - r=mak,scunnane
Differential Revision: https://phabricator.services.mozilla.com/D205025
2024-04-04 12:25:52 +00:00
James Teow
d668670dc8 Bug 1887227 - Allow the ability to reset CategorizationRecorder from tests - r=scunnane
When turning the feature gating pref on and running tests, it revealed one oversight:
When test files are run and we reset telemetry, we don't properly unload the internal
state of CategorizationRecorder. Doing so means that we can keep the "threshold" variable
in a non-zero state despite reseting all telemetry, which can cause submit() to fire when
we don't expect it.

Differential Revision: https://phabricator.services.mozilla.com/D205507
2024-03-26 17:25:19 +00:00
James Teow
14a4f21eb3 Bug 1887233 - Don't count non-sponsored components as visible ads - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D205570
2024-03-26 01:57:31 +00:00
Stephanie Cunnane
14eeeccf77 Bug 1887686 - Add shopping page property to SERP categorization event. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D205607
2024-03-25 21:31:28 +00:00
Mark Banner
937ef9b2d2 Bug 1878834 - Rename search-telemetry-schema to include v2 and add ui schema tests. r=scunnane
Depends on D205250

Differential Revision: https://phabricator.services.mozilla.com/D205341
2024-03-25 14:06:37 +00:00
Stephanie Cunnane
ae45f2ecde Bug 1872935 - Reconcile original categorization ping patches with updated code. r=jteow,firefox-desktop-core-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D197826
2024-03-22 17:57:09 +00:00
Noemi Erli
10a10f24fb Backed out changeset bc969033195f (bug 1872935) for causing failures in test_ping_submitted.py 2024-03-21 00:56:09 +02:00
Stephanie Cunnane
b44b0f6d2d Bug 1872935 - Reconcile original categorization ping patches with updated code. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D197826
2024-03-20 21:12:19 +00:00
Cristian Tuns
6053c16cee Backed out changeset e38df2dd9c60 (bug 1872935) for causing bc failures in browser_search_glean_serp_event_telemetry_categorization_enabled_by_nimbus_variable.js CLOSED TREE 2024-03-20 16:33:47 -04:00
Stephanie Cunnane
6830f8d52e Bug 1872935 - Reconcile original categorization ping patches with updated code. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D197826
2024-03-20 19:26:39 +00:00
James Teow
e7c6bac76d Bug 1875287 - Extract urls from query param values in links when required - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D201841
2024-03-12 01:31:11 +00:00
Karandeep
d54583fc7d Bug 1849371 - Add cookie banner as a possible serp.ad_impression component and the user choice as one of the serp.engagement targets. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D200862
2024-03-08 11:47:46 +00:00
James Teow
f83afd1a47 Bug 1883811 - Move domain processing into child process - r=scunnane
Fundamentally, there shouldn't be any differences in functionality. I removed the
unit test since it was covered by the browser test and added a couple scenarios.

Differential Revision: https://phabricator.services.mozilla.com/D203706
2024-03-07 15:39:52 +00:00
James Teow
6e3cc010a2 Bug 1882528 - Convert domains to category map to use a different hashing algorithm - r=scunnane
On top of making the conversion, instead of using raw values in the test which can be hard
to parse when making changes, I converted the raw values back to domains and then use a helper
to convert it before passing it to the Map. I also removed the external files and stored the
values inside either head.js or within the test.

Differential Revision: https://phabricator.services.mozilla.com/D203226
2024-03-05 05:22:28 +00:00
Dave Townsend
ff1cd075a5 Bug 1864896: Autofix unused function arguments (search). r=search-reviewers,Standard8
Differential Revision: https://phabricator.services.mozilla.com/D202983
2024-03-01 23:43:52 +00:00
James Teow
771da539f9 Bug 1881675 - Allow the ability to ignore specific urls in the network observer - r=scunnane
In rare cases, we may need to add event listeners in the content process which also trigger
a network event that would be parsed by the network observer. To avoid counting double
engagements, I propose we add a new array of regular expressions that when a URL matches, will
never be parsed by the SERP telemetry logic in the network observer. This would rely on the
content process events listeners to register the engagement event.

I think the regular expression should be specific enough that it shouldn't match something
in a search result lest we ignore a click on it.

Differential Revision: https://phabricator.services.mozilla.com/D202583
2024-02-27 14:05:22 +00:00
James Teow
512aa69979 Bug 1880410 - Clean up parts of search telemetry - r=scunnane
Largely non-functional changes:
- The pagehide callback can live in the top level pagehide event.
- Moving the submitted check into the callback will make it easier to refactor
  the helper that adds event listeners.
- Changing the callbacks to use target as the name will make it easier to
  remember that the target is potentially overwritable.
- waitForPageWithAction can be used in future tests.

Differential Revision: https://phabricator.services.mozilla.com/D202036
2024-02-23 17:16:14 +00:00
James Teow
015c906917 Bug 1878219 - Move the core Categorization Map into a worker - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D201026
2024-02-20 22:15:43 +00:00
James Teow
8c241cb504 Bug 1878062 - Check for a possible content process event before manually registering an engagement event - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D200405
2024-02-07 12:58:25 +00:00
Stephanie Cunnane
1f57cdbb59 Bug 1878394 - Modify inconclusive domains to only have a single category (just 'inconclusive'). r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D200511
2024-02-02 23:44:14 +00:00
James Teow
45c16fd91e Bug 1864214 - Observe actions on single page application search engine providers - r=Standard8
updateTrackingSinglePageApp() contains the core logic for when we observe same document loads.
Based on conditions, it may record telemetry (engagement/abandonment), and dispatch events
to the appropriate SearchSERPTelemetry actor.

SearchSERPTelemetryChild had to be modified to allow for manual dispatching of events, especially
unloading event listeners in the case where a user navigates to a non-default search page, like
Images or Shopping.

One complication in observing clicks is we're storing search pages based on the URL at the time
of load, and then looking them up again using the originURL, but it might be different from
the currentURL due to the way single page apps work. So this could cause confusion if multiple SERPs
are open and different from their originURL, hence the usage of recovering the correct state map by
first using the browser object and defaulting back to the originURL if the WeakMap doesn't
contain any of the browser objects.

I also created a mock SPA html file that when given a search query parameter, shows mock results.
The helper file head-spa.js contains a class that navigates the pages in tests.

The tests are broken out into the following:
- When a single search provider is opened in a single tab
- When a single search provider is has multiple tabs open
- When multiple providers are open at the same time
- Event listeners on in-content elements are unloaded properly when navigating away from the page

Differential Revision: https://phabricator.services.mozilla.com/D193320
2024-01-31 12:51:54 +00:00
Stanca Serban
c109e5d34d Backed out changeset 716bfac5741b (bug 1864214) foir causing webrender debug leakcheck failures. CLOSED TREE 2024-01-29 21:41:29 +02:00
James Teow
a461777d68 Bug 1864214 - Observe actions on single page application search engine providers - r=Standard8
updateTrackingSinglePageApp() contains the core logic for when we observe same document loads.
Based on conditions, it may record telemetry (engagement/abandonment), and dispatch events
to the appropriate SearchSERPTelemetry actor.

SearchSERPTelemetryChild had to be modified to allow for manual dispatching of events, especially
unloading event listeners in the case where a user navigates to a non-default search page, like
Images or Shopping.

One complication in observing clicks is we're storing search pages based on the URL at the time
of load, and then looking them up again using the originURL, but it might be different from
the currentURL due to the way single page apps work. So this could cause confusion if multiple SERPs
are open and different from their originURL, hence the usage of recovering the correct state map by
first using the browser object and defaulting back to the originURL if the WeakMap doesn't
contain any of the browser objects.

I also created a mock SPA html file that when given a search query parameter, shows mock results.
The helper file head-spa.js contains a class that navigates the pages in tests.

The tests are broken out into the following:
- When a single search provider is opened in a single tab
- When a single search provider is has multiple tabs open
- When multiple providers are open at the same time
- Event listeners on in-content elements are unloaded properly when navigating away from the page

Differential Revision: https://phabricator.services.mozilla.com/D193320
2024-01-29 15:04:51 +00:00
James Teow
08822f7135 Bug 1875483 - Remove locale and change app version to only report its major app version - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D199103
2024-01-23 14:02:57 +00:00
James Teow
2099213042 Bug 1871084 - Add a timeout for categorization impressions - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D198592
2024-01-19 01:59:29 +00:00
Stephanie Cunnane
d7cef790a9 Bug 1854196 - Modify categorization logic based on input from Data Science. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D197215
2023-12-28 22:37:41 +00:00
James Teow
8cb8c1b19d Bug 1871083 - Part 3: Add application related information to categorization event - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D197129
2023-12-27 16:16:40 +00:00
James Teow
a241368622 Bug 1871083 - Part 2: Add region and locale to categorization event - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D197128
2023-12-27 16:16:40 +00:00
James Teow
a5b9fe4c46 Bug 1871083 - Part 1: Add search provider details to categorization event - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D197127
2023-12-27 16:16:39 +00:00
James Teow
38da77ef2e Bug 1869064 - Refactor recording a SERP categorization until after the user leaves the SERP - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D196491
2023-12-20 18:12:30 +00:00
James Teow
8f395a07bf Bug 1866548 - Detect clicks on SERP ads and organic results that open new windows - r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D194798
2023-12-01 15:11:04 +00:00
James Teow
71efa2123d Bug 1866546 - Move SERP telemetry click logic into its own helper - r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D194774
2023-12-01 07:11:06 +00:00
James Teow
0853e415e9 Bug 1785104 - Update consumers of search-telemetry-v2 when it is updated - r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D192482
2023-11-13 14:43:53 +00:00
Mark Banner
89ef560a9e Bug 1861676 - Use the new queryParamNames SERP telemetry parameter by default. r=search-reviewers,jteow
Differential Revision: https://phabricator.services.mozilla.com/D191649
2023-11-03 10:35:04 +00:00
Cosmin Sabou
40136d94be Backed out changeset eb4c4b32c59b (bug 1861676) for failures on browser_urlbar_telemetry.js. 2023-11-03 01:49:21 +02:00
Mark Banner
3e1df69ce4 Bug 1861676 - Use the new queryParamNames SERP telemetry parameter by default. r=search-reviewers,jteow
Differential Revision: https://phabricator.services.mozilla.com/D191649
2023-11-02 21:02:01 +00:00
James Teow
f42e52abb9 Bug 1855356 - Allow download of collection attachments to re-occur after failure - r=Standard8
Depends on D189992

Differential Revision: https://phabricator.services.mozilla.com/D190247
2023-10-26 00:45:34 +00:00
James Teow
10d9865ad4 Bug 1851495 - Add boolean to indicate whether a user was private browsing when loading a SERP - r=daleharvey,scunnane
Differential Revision: https://phabricator.services.mozilla.com/D191217
2023-10-18 05:17:36 +00:00
Stephanie Cunnane
d1ae2248d3 Bug 1854200 - Limit how many domains are evaluated in SERP categorization process. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D190511
2023-10-14 01:21:41 +00:00
James Teow
cc76365010 Bug 1856274 - Update DomainToCategoriesMap when an experiment is enrolled - r=scunnane
Differential Revision: https://phabricator.services.mozilla.com/D189992
2023-10-06 19:02:32 +00:00
Stephanie Cunnane
dbdb14f49c Bug 1853757 - Don't report SERP categorization for clients that haven't downloaded Remote Setting attachments. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D188748
2023-09-28 18:11:56 +00:00
Stephanie Cunnane
8d3f762d60 Bug 1846368 - Implement SERP categorization logic. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D187810
2023-09-19 16:22:58 +00:00
James Teow
b026880f58 Bug 1850000 - Add listener and in-memory storage of Search Categorization collection - r=Standard8,leplatrem
Differential Revision: https://phabricator.services.mozilla.com/D186805
2023-09-18 13:42:53 +00:00
James Teow
621a5f3aef Bug 1849728 - Compare URLs in page to the URL being visited in the network process - r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D186681
2023-09-13 23:14:17 +00:00
James Teow
69b07fd670 Bug 1842554 - Enable SERP Telemetry improvements - r=Standard8
Differential Revision: https://phabricator.services.mozilla.com/D186552
2023-09-07 10:18:43 +00:00
Stephanie Cunnane
1c5fdd13a5 Bug 1846357 - Extract domains for basic SERP links. r=jteow
Differential Revision: https://phabricator.services.mozilla.com/D185609
2023-09-05 16:49:53 +00:00