25

@SerViette @joystmp @Vlad
Currently the performance between the default lists and the ones listed by joystmp is identical based on every test I have done. That includes the d3ward test and real world tests on some ad-infested sites such as https://www.bournemouthecho.co.uk/ and https://www.golem.de/. The performance is not at Firefox + uBlock Origin level yet but that is being worked on.

The ultimate goal would be to leverage the native blocker (incredibly fast, good performance and low resource usage) in conjunction with an optional advanced script-based blocking solution for the shortcomings mentioned by @Anatomy5803 — achieving a best of both worlds blocking experience.

To summarize, you should be fine with either block list configuration for now but exciting additions are on the horizon if all goes according to plan 🚀

If there are individuals who are well-versed in all things adblocking on WebKit, feel free to share your insights.

    laiz
    you can see difference if you load the "bypass paywall" filterlist because this now work.
    I agree that a lot of cosmetic scriplet are not being injected at the right moment ( if you reload the second hit mostly work) that is mostly true for cookie banner.
    I'm also in lockdown mode so could be some javascript doesn't work on my side

      SerViette note that this filterlists have different optimizations for ios, for safari macos and for macos adguard app...

        chorizo this is true for mobile for desktop I use the "full filterlist" plus adguard pop ups userscript... you can see the difference is night and day if you go to a "ahrr Netflix" site

        • Vlad replied to this.

          chorizo Can you list your recommendations to use for both mobile and desktop and detail the differences so we can test?

            Vlad Shure! On ios/ipados I use the following lists:

            Base filter
            https://filters.adtidy.org/ios/filters/2_optimized.txt
            Mobile filter
            https://filters.adtidy.org/ios/filters/11_optimized.txt
            Tracking Protection filter
            https://filters.adtidy.org/ios/filters/3_optimized.txt
            Social Media filter
            https://filters.adtidy.org/ios/filters/4_optimized.txt
            Annoyances
            https://filters.adtidy.org/ios/filters/14_optimized.txt
            Badware Risk
            https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/filters/badware.txt
            Paywall
            https://gitflic.ru/project/magnolia1234/bypass-paywalls-clean-filters/blob/raw?file=bpc-paywall-filter.txt
            +Easylist for my language

            system wide I use:
            Lockdown Mode, the HaGeZi’s Pro and HaGeZi’s Threat Intelligence Feeds DNS Blocklists

            The differences between Easy privacy and tracking protection by adguard can be seen on the tracking test you can find at https://coveryourtracks.eff.org

            Easylist:

            AG Tracking Protection:

            On MacOs:

            Base filter https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_2_Base/filter.txt

            Tracking Protection filter https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_3_Spyware/filter.txt

            URL Tracking filter (note that this isn’t suitable for iOs)
            https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_17_TrackParam/filter.txt

            Social media filter https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_4_Social/filter.txt

            Annoyances filter https://raw.githubusercontent.com/AdguardTeam/FiltersRegistry/master/filters/filter_14_Annoyances/filter.txt

            Anti Adblock Warning
            https://easylist-downloads.adblockplus.org/antiadblockfilters.txt

            plus:

            • bypass paywall same as above
            • uBlock Badware risk same as above
            • Language specific filters

            I use violentmonkey as userscript extension with the following userscripts:

            AG Pop Up Blocker:
            https://userscripts.adtidy.org/release/popup-blocker/2.5/popupblocker.user.js

            Bypass Paywall:
            https://gitflic.ru/project/magnolia1234/bypass-paywalls-clean-filters/blob/raw?file=userscript/bpc.en.user.js

            System wide also here:
            Lockdown Mode, the HaGeZi’s Pro and HaGeZi’s Threat Intelligence Feeds DNS Blocklists

            is not smooth and error free as using the same filterlists as with the AG app, also because the AG app uses more userscripts on top of what I’ve listed, BUT, I really feel the step forward of Orion in comparision to the previous rule converter library

            • laiz replied to this.

              Note before adding AG Annoyances as default, it can be broken into more smaller and dedicated lists since maybe someone might need more fine grained control.
              From AG docs:

              Annoyances filter blocks irritating elements on web pages. View rules. Includes the following AdGuard filters (all of them can be enabled separately from the Annoyances filter):

              Cookie Notices blocks cookie notices on web pages. View rules
              Popups blocks all kinds of pop-ups that are not necessary for websites' operation. View rules
              Mobile App Banners blocks banners that promote mobile apps of websites. View rules
              Widgets blocks third-party widgets: online assistants, live support chats, etc. View rules
              Other Annoyances blocks elements that do not fall under the popular categories of annoyances. View rules

                4 days later

                @Vlad the implementation of using the rules converter library clearly isn't done, because it doesn't seem to do scriptlet injection. Case in point, YouTube ads still appear.

                • laiz replied to this.

                  Anatomy5803
                  That would be #7362
                  Nevertheless, I agree that this feature isn't exactly revolutionary until scriptlet support is also added. Based on the discussions I've been a part of, including custom support for that is on the roadmap but is not yet being worked on (the focus is currently on other adblocking features and fully supporting uBO).

                    chorizo
                    I tested the following on macOS using a fresh profile on Orion Version 0.99.129.3-rc (WebKit 621.1.2.111.4)
                    :


                    This configuration left a lot to be desired on my setup.

                    For comparison, here's the default configuration Orion currently ships with.

                    • 100 % on d3ward test
                    • ads also get through on https://www.golem.de (this requires a German blocklist which AdGuard enables automatically. Curiously uBlock Origin on FF blocks everything without needing to enable German lists.
                    • EFF
                    • https://www.bournemouthecho.co.uk ads were blocked as expected

                    Could you clarify whether your experience is the same, or if I am missing something?

                    The inclusion of that many blocklists by default also comes with the risk of some sites breaking, as was the case back when Orion shipped with a preconfigured blocklist that was very effective but came with the downside of some cookie banners being blocked, rendering some websites (e.g. reddit) unusable until the user toggled the content blocker off, accepted cookies, and reenabled the content blocker. That is why the current default config is good enough, and that should also be the goal for Orion's default blocker configuration.

                    Moreover, including bypass paywalls lists by default is not something I'd recommend. Instead Orion should take a neutral stance, and let people pick whether they'd be willing to pay for content or not read it at all (or install workarounds at the cost of their own time).

                      laiz Hello, we are confusing the topics: I am not recommending any default filter lists, the thread is about the rules converter library and whether it can now apply filter lists without the ABP syntax.
                      I was asked to share my configuration to highlight the differences compared to the previous rule converter library and so I did.
                      For me, whatever the default filterlists are makes no difference as long as I can apply the ones of my choice especially on mobile, if on desktop I can have work arounds on mobile I'm more limited (adguard cannot filter other browsers other than safari in ios).
                      This new library may do better filtering (IMHO) if all rules were applied correctly. Some non ABP rules are applied, the bypass paywall is always mentioned because is easier to see if is working or not, cookie banner never get blocked in every configuration for example..
                      I Know the full potential of those filterlists because I see the result of them being applied thru the AG app so would be nice if Orion can apply them on his own natively without relying on extensions.

                      Going back to the comparision of the filterlists I must point out that from your screenshot I see that many rules have not been loaded; there should be a little over 200,000, and you have many fewer.

                      I created 2 new profiles with the latest Orion RC.

                      The blue profile is Orion default.
                      The red profile is Orion with the filter lists I mentioned above.
                      To try to be more neutral as possible, I did not load any specific language filter, no extensions installed and I disabled DNS filtering for the entire system.

                      here wins the default configuration:

                      here wins the alternative configuration:

                      unpaywall:

                      golem.de is half filtered in different ways:

                      golem.de on mobile:

                      youtube:
                      wins the default configuration that skips the ad while the alternative configuration obscures it by making you wait before playing the video.

                      so has stated above there are some script that are not applied but others do, the more notable the BPC.

                        chorizo
                        Above and beyond effort, thank you!

                        I retried adding the lists and was now able get 233k active rules as well. I am not sure what was the issue yesterday.

                        I am not recommending any default filter lists, the thread is about the rules converter library and whether it can now apply filter lists without the ABP syntax.

                        A week ago Vlad asked for your recommendations on filter lists for macOS and iOS, to which you replied with the lists I tested yesterday. That is where my assumption of changing the defaults came from. I understand now that it wasn't your intention and you instead focused on detailing the differences between your setup and the current Orion defaults. I apologize for the confusion caused.

                        If I've understood correctly, we are all in agreement that the library has indeed been implemented and we are now waiting for scriptlet support so the adblocking capabilities of Orion can be taken to the next level. #7362 is the suggestion for scriptlet support.

                        Moreover, the consensus seems to be that there is no reason to touch the default Orion lists at the moment. They are sensible and work for the vast majority of users. Advanced users can configure their filter lists as they see fit.

                        PS: I don't use youtube but for what is worth this is my hompage don't Know if differ from yours:

                        here's the ad obscured but not skipped,wait till the end and the video starts:

                          9 days later

                          Once Orion implements scriptlet support (called advanced js by them), like WebShield or wBlock does like Safari, YouTube ads will be blocked. Also you don't need optimized lists for Orion since I believe each list is a content blocker & advanced js is run in a js file not through content blocking.

                          The lists you should use could be the default lists in WebShield.

                          Off the top of my head, these should be enabled by default:
                          AdGuard Base
                          AdGuard Tracking
                          EasyPrivacy

                          I'm developing WebShield which is an adblocker for Safari, so if you have any questions regarding lists, please lmk. Respectfully, While @yokoffing's suggestions were better than just plain EasyList, Hagezi lists are meant primarily for DNS, have no cosmetic or especially scriptlet rules. You can get far better results by using the above lists as defaults. You don't need the optimized lists either I believe for the Orion implementation.

                            Anatomy5803 doesn't AG tracking protection include/is based on easy privacy?
                            aren't the two redundant?
                            I genuinely ask since I always thought that AG tracking protection has easy privacy inside, also is like 4 times bigger?

                            the ios optimized filters are what adguard use in they ios app, they have lesser rules and I think speed up the browsing...

                            with the same set of filterlists i have circa 230000 rules on desktop but circa 98000 on mobile

                              Anatomy5803 The normal filter lists include all rules, whereas optimised lists include most frequently used rules/domains. Lesser rules leads to fewer processing and better performance.

                              Scriptlets have nothing to do with optimised lists, since both lists include scriptlets.

                                chorizo AG Tracking Protection isn't based on EasyPrivacy to my memory. AG Base is based on EasyList, thus you don't need to enable AG Base & EasyList, since AG Base already has EasyList. Optimized lists exist because Safari ContentBlockers have a limit of 160k rules IIRC, and each category is 1 content blocker. Since it seems like Orion is using 1 content blocker for each list, using optimized lists won't matter depending upon what the limit of a WebKit Content Blocker is, or if its the same as Safari, which would be 160k per each list.

                                @joystmp Yes, that is true that lesser rules leads to fewer processing, but that's only at conversion time, since WebKit-content blocking rules are very efficient & performant since they apply before network load to my memory. Scriptlets would be the only performance issue since they're processed in realtime through javascript.

                                Scriptlets and cosmetic filtering isn't in Hagezi, that's what I was referring to.

                                  3 months later

                                  Anatomy5803 hi, do you noticed content blocking regression with this last update on mobile?
                                  1.3.18

                                    No one is typing