Skip to content Skip to sidebar Skip to footer

How To Get All Text Between Just Two Specified Tags Using BeautifulSoup?

html = ''' ... all ( iterable )

Solution 1:

An iterative approach.

from BeautifulSoup import BeautifulSoup as bs
from itertools import takewhile, chain

def get_text(html, from_tag, until_tag):
    soup = bs(html)
    for big in soup(from_tag):
        until = big.findNext(until_tag)
        strings = (node for node in big.nextSiblingGenerator() if getattr(node, 'text', '').strip())
        selected = takewhile(lambda node: node != until, strings)
        try:
            yield ''.join(getattr(node, 'text', '') for node in chain([big, next(selected)], selected))
        except StopIteration as e:
            pass

for text in get_text(html, 'big', 'a'):
    print text

Solution 2:

I would avoid nextSibling, as from your question, you want to include everything up until the next <a>, regardless of whether that is in a sibling, parent or child element.

Therefore I think the best approach is to find the node that is the next <a> element and loop recursively until then, adding each string as encountered. You may need to tidy up the below if your HTML is vastly different from the sample, but something like this should work:

from bs4 import BeautifulSoup
#by taking the `html` variable from the question.
html = BeautifulSoup(html)
firstBigTag = html.find_all('big')[0]
nextATag = firstBigTag.find_next('a')
def loopUntilA(text, firstElement):
    text += firstElement.string
    if (firstElement.next.next == nextATag):             
        return text
    else:
        #Using double next to skip the string nodes themselves
        return loopUntilA(text, firstElement.next.next)
targetString = loopUntilA('', firstBigTag)
print targetString

Solution 3:


Solution 4:

>>> from BeautifulSoup import BeautifulSoup as bs
>>> parsed = bs(html)
>>> txt = []
>>> for i in parsed.findAll('big'):
...     txt.append(i.text)
...     if i.nextSibling.name != u'a':
...         txt.append(i.nextSibling.text)
...
>>> ''.join(txt)
u'(iterable)'

Post a Comment for "How To Get All Text Between Just Two Specified Tags Using BeautifulSoup?"