Skip to content

smileboywtu/colly

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Colly

Scraping Framework for Gophers

Example:

import (
	"fmt"
	"github.com/asciimoo/colly"
)

func main() {
	c := colly.NewCollector()

    // Find and visit all links
	c.OnHTML("a", func(e *colly.HTMLElement) {
		link := e.Attr("href")
		fmt.Println(link)
		c.Visit(e.Request.AbsoluteURL(link))
	})

	c.Visit("https://en.wikipedia.org/")
}

See examples folder for more detailed examples.

Features

  • Clean API
  • Cookies and session handling
  • Sync/async/parallel scraping
  • Fast (>1k request/sec on a single core)

Bugs

Bugs or suggestions? Visit the issue tracker or join #colly on freenode

About

Fast and Elegant Scraping Framework for Gophers

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Go 100.0%