@@ -301,31 +438,137 @@ When website owners implement structural changes like
The selector will no longer function and your code needs maintenance. That's where Scrapling auto-matching feature comes into play.
```python
+from scrapling import Adaptor
# Before the change
-page = Adaptor(page_source, url='example.com', auto_match=True)
+page = Adaptor(page_source, url='example.com')
element = page.css('#p1' auto_save=True)
if not element: # One day website changes?
- element = page.css('#p1', auto_match=True) # Still finds it!
+ element = page.css('#p1', auto_match=True) # Scrapling still finds it!
# the rest of the code...
```
> How does the auto-matching work? Check the [FAQs](#-enlightening-questions-and-faqs) section for that and other possible issues while auto-matching.
+#### Real World Scenario
+Let's use a real website as an example and use one of the fetchers to fetch its source. To do this we need to find a website that will change its design/structure soon and take a copy of its source then wait for the website to do the change. Of course, that's nearly impossible to know unless I know website's owner but that will make it a staged test haha.
+
+To solve this issue, I will use [The Web Archive](https://archive.org/)'s [Wayback Machine](https://web.archive.org/). Here is a copy of [StackOverFlow's website in 2010](https://web.archive.org/web/20100102003420/http://stackoverflow.com/), pretty old huh?Let's test if the automatch feature can extract the same button in the old design from 2010 and the current design using the same selector :)
+
+If I want to extract the Questions button from the old design I can use a selector like this `#hmenus > div:nth-child(1) > ul > li:nth-child(1) > a` this selector is too specific because it was generated by Google Chrome.
+Now let's test the same selector in both versions
+```python
+>> from scrapling import Fetcher
+>> selector = '#hmenus > div:nth-child(1) > ul > li:nth-child(1) > a'
+>> old_url = "https://web.archive.org/web/20100102003420/http://stackoverflow.com/"
+>> new_url = "https://stackoverflow.com/"
+>>
+>> page = Fetcher(automatch_domain='stackoverflow.com').get(old_url, timeout=30).adaptor
+>> element1 = page.css_first(selector, auto_save=True)
+>>
+>> # Same selector but used in the updated website
+>> page = Fetcher(automatch_domain="stackoverflow.com").get(new_url).adaptor
+>> element2 = page.css_first(selector, auto_match=True)
+>>
+>> if element1.text == element2.text:
+... print('Scrapling found the same element in the old design and the new design!')
+'Scrapling found the same element in the old design and the new design!'
+```
+Note that I used a new argument called `automatch_domain`, this because for Scrapling these are two different URLs not the website so it isolates their data. To tell Scrapling they are the same website, we the pass the domain we want to use for saving auto-match data for them both so Scrapling doesn't isolate them.
+
+In real world scenario, the code will be the same expect it will use the same URL for both requests so you won't need to use `automatch_domain` argument. This is the closest example I can give to real world cases so I hope it didn't confuse you :)
+
**Notes:**
-1. Passing the `auto_save` argument without setting `auto_match` to `True` while initializing the Adaptor object will only result in ignoring the `auto_save` argument value and the following warning message
+1. For the two examples above I used one time the `Adaptor` class and the second time the `Fetcher` class just to show you that you can create the `Adaptor` object by yourself if you have the source or fetch the source using any `Fetcher` class then it will create the `Adaptor` object for you on the `.adaptor` property.
+2. Passing the `auto_save` argument with the `auto_match` argument set to `False` while initializing the Adaptor/Fetcher object will only result in ignoring the `auto_save` argument value and the following warning message
```text
Argument `auto_save` will be ignored because `auto_match` wasn't enabled on initialization. Check docs for more info.
```
This behavior is purely for performance reasons so the database gets created/connected only when you are planning to use the auto-matching features. Same case with the `auto_match` argument.
-2. The `auto_match` parameter works only for `Adaptor` instances not `Adaptors` so if you do something like this you will get an error
+3. The `auto_match` parameter works only for `Adaptor` instances not `Adaptors` so if you do something like this you will get an error
```python
page.css('body').css('#p1', auto_match=True)
```
because you can't auto-match a whole list, you have to be specific and do something like
```python
- page.css('body')[0].css('#p1', auto_match=True)
+ page.css_first('body').css('#p1', auto_match=True)
```
+### Find elements by filters
+Inspired by BeautifulSoup's `find_all` function you can find elements by using `find_all`/`find` methods. Both methods can take multiple types of filters and returns all elements in the pages that all these filters apply to.
+
+* To be more specific:
+ * Any string passed is considered a tag name
+ * Any iterable passed like List/Tuple/Set is considered an iterable of tag names.
+ * Any dictionary is considered a mapping of HTML element(s) attributes name and attributes values.
+ * Any regex patterns passed are used as filters
+ * Any functions passed are used as filters
+ * Any keyword argument passed is considered as a HTML element attribute with its value.
+
+So the way it works is after collecting all passed arguments and keywords, each filter passes its results to the following filter in a waterfall-like filtering system.
+
It filters all elements in the current page/element in the following order:
+
+1. All elements with the passed tag name(s).
+2. All elements that matches all passed attribute(s).
+3. All elements that matches all passed regex patterns.
+4. All elements that fulfills all passed function(s).
+
+Note: The filtering process always starts from the first filter it finds in the filtering order above so if no tag name(s) are passed but attributes passed, the process starts from that layer and so one. **But the order in which you pass the arguments doesn't matter.**
+
+Examples to clear any confusion :)
+
+```python
+>> from scrapling import Fetcher
+>> page = Fetcher().get('https://quotes.toscrape.com/').adaptor
+# Find all elements with tag name `div`.
+>> page.find_all('div')
+[