1. Halo Guest, pastikan Anda selalu menaati peraturan forum sebelum mengirimkan post atau thread baru.

[Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keyword

Discussion in 'Freebies' started by mrlemonade, Dec 25, 2010.

Tags:
Page 1 of 2
  1. mrlemonade

    mrlemonade Hero

    Joined:
    Sep 20, 2007
    Messages:
    548
    Likes Received:
    55
    SphinxBot v0.3a *updated 12/25/2010 08:09*

    Apaan sih SphinxBot?

    SphinxBot versi public ini adalah bot yg gunanya akan scraping ke salah satu website source yang udah di tentukan dengan keyword yang di inginkan, lalu dia akan coba untuk figure out title dan content dari masing2 post yg ditemukan secara otomatis dan akan ngasih output berupa rss yang siap dipake.

    saat ini ada 2 4 source yang direlease untuk public yaitu:

    * Google News
    * Google Blog Search
    * Bing News
    * Bing Web Search


    note: karena semuanya serba auto, mungkin ada beberapa element content dari situs asal yg mengganggu ngikut ke scrap.

    tkp:
    Code:
    http://crotz.net/lemonade/sphinxbot.php
    contoh hasil rss:
    Code:
    http://crotz.net/lemonade/sphinxbot.php?keyword=aff+final&source=2&lKey=kljkl412klh3k1hadflasdf&pisang=http%3A%2F%2Fkambing.freevar.com%2Fpisang.php
http://crotz.net/lemonade/sphinxbot.php?keyword=adsense+indonesia&source=2&lKey=kljkl412klh3k1hadflasdf&pisang=http%3A%2F%2Fkambing.freevar.com%2Fpisang.php
    your own - pisang.php
    Code:
    <?php

// key ini dibuat supaya kalo misalnya pisang.php diketahuin sama orang, tapi kan key nya yg tau cuman kita masing2 aja.
$myLocalKey = 'masukin_key_nya_terserah_apa_aja';

echo doPisang($myLocalKey);


///////////////////////////////////////////////////////////////////////

function doPisang($myLocalKey) {
if(!empty($_POST['lKey']) && !empty($_POST['url'])) {
  $lKey = urldecode($_POST['lKey']);
  $url = base64_decode(rawurldecode(urldecode($_POST['url'])));

  $url = 'http://'.$url;

  if($lKey == $myLocalKey) {
    return ambil($url);
  }
  else
    return "key didnt match";
}
}

function ambil($url) {

        $ch = curl_init();
            curl_setopt($ch, CURLOPT_URL, $url);
                curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
                curl_setopt($ch, CURLOPT_USERAGENT, "SphinxBot v0.2a");
                curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
                curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 20);
                curl_setopt($ch, CURLOPT_TIMEOUT, 30);

                $res = curl_exec($ch);
                curl_close($ch);
                return $res;
}
     
    Last edited: Dec 25, 2010
    mrlemonade, Dec 25, 2010
    #1
    D'Harisy, mukh17 and littlethinker like this.
  2. teguhaditya

    teguhaditya Super Hero

    Joined:
    Jan 23, 2008
    Messages:
    7,503
    Likes Received:
    1,418
    Location:
    _ ▂ ▃ ▅ ▆ █
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    source code nya ada gak om?
     
    teguhaditya, Dec 25, 2010
    #2
  3. mrlemonade

    mrlemonade Hero

    Joined:
    Sep 20, 2007
    Messages:
    548
    Likes Received:
    55
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    waduuh.. pertanyaan yang sulit nih bro hehehuehuhe.. :p
    source codenya ada tapi untuk sekarang masih closed source dulu :)
     
    mrlemonade, Dec 25, 2010
    #3
  4. caktakim

    caktakim Super Hero

    Joined:
    Dec 26, 2009
    Messages:
    2,644
    Likes Received:
    168
    Location:
    Tengah dalan
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    Punya ente yak OM :D
     
    caktakim, Dec 25, 2010
    #4
  5. mrlemonade

    mrlemonade Hero

    Joined:
    Sep 20, 2007
    Messages:
    548
    Likes Received:
    55
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    iyaa part of the code punya gue bro :)
     
    mrlemonade, Dec 25, 2010
    #5
  6. ensept

    ensept Hero

    Joined:
    Dec 2, 2005
    Messages:
    654
    Likes Received:
    6
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    ikutan coba ya..tq b4
     
    ensept, Dec 25, 2010
    #6
  7. littlethinker

    littlethinker Super Hero

    Joined:
    Nov 7, 2010
    Messages:
    1,302
    Likes Received:
    42
    Location:
    Palangkaraya
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    mantau ya om, newbie ngak ngerti yg bginian, *ayolah, dbka aja om source codenya,:peace: insyaallah barokah,
     
    littlethinker, Dec 25, 2010
    #7
  8. genggamcahaya

    genggamcahaya Hero

    Joined:
    May 12, 2010
    Messages:
    607
    Likes Received:
    28
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    khusus buat wp yak?
     
    genggamcahaya, Dec 25, 2010
    #8
  9. ensept

    ensept Hero

    Joined:
    Dec 2, 2005
    Messages:
    654
    Likes Received:
    6
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    "Pengen punya key and pisang.php sendiri buat dipake bareng sphinxbot? well.. contact me. "

    contact me nya kemana?
     
    ensept, Dec 25, 2010
    #9
  10. mrlemonade

    mrlemonade Hero

    Joined:
    Sep 20, 2007
    Messages:
    548
    Likes Received:
    55
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    kontak ke thread ini maksud nya huehue.. lupa di taro live link nya tadi.
    pisang.php nya cuman berfungsi sebagai proxy aja untuk ngambil halaman2 source nya. semua parsingnya tetep dilakukan di server gue.

    kenapa gue bikin kyk gitu, soalnya kalo ga pake gini2an, nanti kalo di abused makenya gak kira2, nanti gue yg ga tau apa2 kena dari macem2 pihak dong hihihihi.. :D

    ---------- Post added at 07:31 AM ---------- Previous post was at 07:30 AM ----------
    download pisang.php nya aja terus edit key nya sendiri di pisang.php untuk dipake disana, itu bakalan tetep online kok bro.

    *uda gue update di post awal*
     
    mrlemonade, Dec 25, 2010
    #10
  11. aditiya1920

    aditiya1920 Super Hero

    Joined:
    Apr 16, 2010
    Messages:
    1,863
    Likes Received:
    43
    Location:
    Stay Here XTidu, Bojonegoro
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    G mudeng nieh bos, makeknya gmna yah? klo abis edit2..

    terus diapain lagi bro..
     
    aditiya1920, Dec 25, 2010
    #11
  12. mrlemonade

    mrlemonade Hero

    Joined:
    Sep 20, 2007
    Messages:
    548
    Likes Received:
    55
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    hasil rss yg udah di generate cara pake nya di auto post ke bot loe dengan cara:

    1. rss2post plugin semacem wpomatic, feedwordpress, dll
    2. dan lain2.

    sorry gue cuman menyediakan resource bagi yg butuh, kalau mau ngolah datanya gimana caranya, mungkin bisa tanya2 sama master2 blogger disini yg suka maen autoblog atau agc. thx.
     
    mrlemonade, Dec 25, 2010
    #12
  13. pikun

    pikun Hero

    Joined:
    Jul 5, 2010
    Messages:
    514
    Likes Received:
    91
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    keren.... :)
    kalo saya lihat ada yang kurang dengan free service yang bos ts tawarkan:
    - crotz.net lumayan lemot, pertanyaannya adalah crotz.net mampu melayani berapa request per second?
    - full rss yang dihasilkan kok ngga ada picture-nya ya, padahal di source webnya ada picturenya....

    sebagai perbandingan, ada free service juga dari fivefilters.org (bahkan free source code). Full RSS yang dihasilkan sudah dilengkapi dengan picture (jadi benar-benar full scrapping).
    berikut ini perbandingan (pisang to pisang) full rss yang dihasilkan oleh fivefilters.org dan crotz.net dengan source google blog search dan kata kunci "indonesia":
    source rss: _http://blogsearch.google.com/blogsearch_feeds?hl=en&q=indonesia&ie=utf-8&num=10&output=rss
    fivefilters.org: _http://fivefilters.org/content-only/makefulltextfeed.php?url=http%3A%2F%2Fblogsearch.google.com%2Fblogsearch_feeds%3Fhl%3Den%26q%3Dindonesia%26ie%3Dutf-8%26num%3D10%26output%3Drss&max=4
    crotz.net: _http://crotz.net/lemonade/sphinxbot.php?keyword=indonesia&source=2&lKey=kljkl412klh3k1hadflasdf&pisang=http%3A%2F%2Fkambing.freevar.com%2Fpisang.php

    nice work dude :) keep rockin'
     
    Last edited: Dec 25, 2010
    pikun, Dec 25, 2010
    #13
  14. mrlemonade

    mrlemonade Hero

    Joined:
    Sep 20, 2007
    Messages:
    548
    Likes Received:
    55
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    thx buat saran n komentar nya. yap memang masih belom sempurna. tapi setidaknya memudahkan bagi yang ingin pake service nya tanpa harus copas url blogsearch dll.
    but atleast im start sharing and not selling. biarpun hanya berupa service dan bukan source code :)

    ps: pisang.php nya si kambing ada di free hosting.
     
    mrlemonade, Dec 25, 2010
    #14
  15. Berantakan

    Berantakan Ads.id Pro

    Joined:
    Aug 4, 2010
    Messages:
    470
    Likes Received:
    7
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    iya klo, mau pake fivefilters,scriptnya ada tuh di bhw.
     
    Berantakan, Dec 25, 2010
    #15
  16. masecho

    masecho Ads.id Pro

    Joined:
    May 6, 2010
    Messages:
    473
    Likes Received:
    10
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    kayaknya bagus nih nmbrung disini banyak pada kasih tips para master2...... bookmark ya....
     
    masecho, Dec 25, 2010
    #16
  17. mrlemonade

    mrlemonade Hero

    Joined:
    Sep 20, 2007
    Messages:
    548
    Likes Received:
    55
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    *update* nambah 2 link pisang.php, abis gue liat udah mulai banyak yg pake tp ga ada yg mo download and pasang pisang.php nya di server sendiri huhuhuhu.. T_T
     
    mrlemonade, Dec 25, 2010
    #17
  18. W4nzz

    W4nzz Super Hero

    Joined:
    Dec 12, 2008
    Messages:
    1,342
    Likes Received:
    94
    Location:
    Kota Billboard (Medan)
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    masi binggung makenya.. tapi tq dah share :D
     
    W4nzz, Dec 26, 2010
    #18
  19. mrlemonade

    mrlemonade Hero

    Joined:
    Sep 20, 2007
    Messages:
    548
    Likes Received:
    55
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    tinggal masukin keyword, setting pisang.php dan key nya.
    terus bikin wordpress, pake plugin wpomatic.

    masukin rss hasil auto generate nya ke wpomatic.

    jadi deh auto blog yg simple.
     
    mrlemonade, Dec 26, 2010
    #19
  20. ooND

    ooND Super Hero

    Joined:
    Jul 24, 2010
    Messages:
    5,707
    Likes Received:
    1,118
    Location:
    Wonosobo, Jawa Tengah, Indonesia, Indonesia
    Re: [Free] SphinxBot - Google Blog Search / Google News - Full Rss Scraper with Keywo

    :lol: kirain cmn ane yg bingung,,, :lol: udah bolak balik baca tetep gag ngeh,,, :peace: nasib nyubi
     
    ooND, Dec 26, 2010
    #20
Page 1 of 2

Share This Page