On: Show HN: Voyager – write a web crawler/scraper as a state machine in Rust

[matsche]

I haven't used scraper too much. I personally find the predicate approach of select.rs [0] easier to use. However in this case the selector approach just made more sense. Standalone html5ever can be a bit cumbersome to work with directly, scraper is basically an implementation of the html5ever's `TreeSink` trait, where as `select.rs` uses the hmtl5ever `RcDom` to parse the document but stores it in a more convenient way. If you look for a minimal approach you should at select.rs which basicially only depends on html5ever

[0] https://github.com/utkarshkukreti/select.rs

[cljacoby → matsche]

Hey just FYI, when you run the hackernews and explore examples without enabling the tokio feature flag, you get compilation errors about undeclared types for all the tokio stuff. I think these examples just need entires in the Cargo.toml requiring the tokio feature like the reddit example.

Once I add the tokio feature, they all run as expected.

[matsche → cljacoby]

Thanks, you're right. I fixed it just now by also requiring the tokio feature for all the tests in the Cargo.toml.