1. Welcome to Chandoo.org Forums. Short message for you

    Hi Guest,

    Thanks for joining Chandoo.org forums. We are here to make you awesome in Excel. Before you post your first question, please read this short introduction guide. When posting or responding to questions please remember our values at Chandoo.org are: Humility, Passion, Fun, Awesomeness, Simplicity, Sharing Remember that we have people here for whom English is not there first language and we need to allow for this in our dealings.

    Yours,
    Chandoo
  2. Hi All

    Please note that at the Chandoo.org Forums there is Zero Tolerance to Spam

    Post Spam and you Will Be Deleted as a User

    Hui...

  3. When starting a new post, to receive a quicker and more targeted answer, Please include a sample file in the initial post.

Expecting someone to help me create a facebook bot

Discussion in 'The Vault' started by shahin, Mar 10, 2017.

  1. shahin

    shahin Member

    Messages:
    342
    Hi there! Good afternoon. I tried to make a facebook bot to parse the profile links. However, it signs in and parses the content of left-sided bar inconsistently. I can't go further. Could anyone point me in the right direction so that i can rectify my mistakes I've made already in my code and move on. Here is the code:

    Code (vb):

    Sub FBbot()
    Dim http As New MSXML2.XMLHTTP60, html As New HTMLDocument
    Dim topics As Object, topic As Object
    Dim x As Long,strdata As String

    x = 2

    strdata = "email=sth.com&pass=xxx"
    http.Open "POST", "https://www.facebook.com/login.php?login_attempt=1&lwv=110", False
    http.setRequestHeader "Content-type", "application/x-www-form-urlencoded"
    http.send strdata

    html.body.innerHTML = http.responseText

    Set topics = html.getElementsByClassName("_li")(0).getElementsByTagName("a")

    For Each topic In topics
        Cells(x, 1) = topic.innerText
        x = x + 1
    Next topic
    End Sub
     
    Last edited: Mar 10, 2017
  2. Hui

    Hui Excel Ninja Staff Member

    Messages:
    10,455
    I can't really help but as I rarely scrape web sites and have never scraped facebook

    In Facebook right click on what you want and Inspect Element

    Make sure that the Topic and Innertext are the correct fields

    You may also want to see if there are different view panes/windows that you need to look through (I'm not sure of the correct term)
    ThrottleWorks and shahin like this.
  3. shahin

    shahin Member

    Messages:
    342
    Thanks sir for your kind reply. I'm trying to find that out also where I'm making mistakes.
  4. shahin

    shahin Member

    Messages:
    342
    I've got it working with the below code. It is necessary to do one thing manually, though. After running this script, when facebook page opens up, there will be a notification popping up mentioning "Allow" or "Block". Wait a little while to get the buffering done and then Just click on "Allow". That's it. Btw, before running this script make sure "Selenium Type Library" has been added in the reference.

    Code (vb):

    Sub Facebook_crawler()
    Dim driver As New WebDriver
    Dim posts As Object, post As Object

    With driver
        .Start "chrome", "https://www.facebook.com"
        .get "/login.php?login_attempt=1&lwv=111"

        .FindElementByXPath("//*[@id='email']").SendKeys ("someEmail") --Fill in according to your email
        .FindElementByXPath("//*[@id='pass']").SendKeys ("password") --Fill in with your password
        .FindElementByXPath("//*[@id='loginbutton']").Click
        .Wait 800
        .FindElementByXPath("//a[@id='findFriendsNav']").Click
        .Wait 500
    End With

    For x = 0 To 50
        driver.ExecuteScript "window.scrollTo(0, document.body.scrollHeight);"
        driver.Wait 500
    Next x

    Set posts = driver.FindElementsByXPath("//div[contains(@class,'friendBrowserNameTitle')]/a")

    For Each post In posts
        i = i + 1
        Cells(i, 1) = post.Text
    Next post
    End Sub
     
    Last edited: Jul 6, 2017
    ThrottleWorks likes this.
  5. shahin

    shahin Member

    Messages:
    342
    Forgot to mention that the above crawler is able to scrape the names of friend request by going to the target page and scrolling down to a certain position.
  6. Hui

    Hui Excel Ninja Staff Member

    Messages:
    10,455
    Have you tried this in Excel 2016+
    upload_2017-7-6_21-19-16.png
    ThrottleWorks and shahin like this.
  7. shahin

    shahin Member

    Messages:
    342
    Thanks sir Hui, for showing me a new way of doing stuffs. Is it power query? I heard that it can do impossible things. Anyways, I use excel 2013.
    ThrottleWorks likes this.
  8. Hui

    Hui Excel Ninja Staff Member

    Messages:
    10,455
    Have a look just in case it's there
    ThrottleWorks likes this.
  9. shahin

    shahin Member

    Messages:
    342
    Found the solution. Now this scraper is able to harvest all the names from the friend list without manual intervention, as in clicking the "Allow" or "Block" notification. It can automate the whole operation. Don't forget to add "Selenium Type Library" in the reference library before execution. Here is the code:
    Code (vb):

    Sub Test_stth()
    Dim driver As New WebDriver

    With driver
        .SetPreference "profile.default_content_setting_values.notifications", 2  'This is the line which is able to block the notification
       .Start "chrome", "https://www.facebook.com"
        .get "/login.php?login_attempt=1&lwv=111"

        .FindElementByXPath("//*[@id='email']").SendKeys ("username")     'Input the username
       .FindElementByXPath("//*[@id='pass']").SendKeys ("password")   'Input the password
       .FindElementByXPath("//*[@id='loginbutton']").Click
        .Wait 800
        .FindElementByXPath("//a[@id='findFriendsNav']").Click
        .Wait 500
    End With

    For x = 0 To 50
        driver.ExecuteScript "window.scrollTo(0, document.body.scrollHeight);"
        driver.Wait 500
    Next x

    Set posts = driver.FindElementsByXPath("//div[contains(@class,'friendBrowserNameTitle')]/a")

    For Each post In posts
        i = i + 1
        Cells(i, 1) = post.Text
    Next post
    End Sub
     
    <Edited as requested by user>
    Last edited by a moderator: Jul 18, 2017 at 3:15 PM
    NARAYANK991 likes this.
  10. shahin

    shahin Member

    Messages:
    342
    As I don't have enough privilege to rectify a certain line in the pasted code above, I'm pasting below the modified line. ID should be [@id='email'] not [@id='username']. Now I suppose You don't find any trouble getting the harvested results.

    Code (vb):

    .FindElementByXPath("//*[@id='email']").SendKeys ("username")    'Input the username

Share This Page