

Show HN: Make your AngularJS, EmberJS, or BackboneJS website crawlable by Google - chaddeshon

http:&#x2F;&#x2F;www.BromBone.com<p>I posted “Tell HN: My Web App has 13 Users” 3 months ago (https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=5386966). Over time, it grew to 150 (free) users, but retention was pretty much zero. I was trying to cast a wide net and offer a generic “headless browser as a service”. It turned out I had make a service that was a little helpful to many people, but didn&#x27;t really solve anyone&#x27;s problem completely.<p>I finally took the advise offered in that thread. I focused on a problem I understood, and solved it completely instead of just offering a helpful tool. Instead of a “hosted headless browser” BromBone is now a complete solution for people who want their javascript websites indexed by Google.<p><i></i><i></i><i></i><p>BromBone.com - The javascript SEO problem has been solved.<p>Google can&#x27;t index javascript driven webpages. The accepted solutions have been:
1. Run your own PhantomJS server to render snapshots and serve the snapshots to Google (a real pain).
2. Accept that your pages won&#x27;t be listed on Google.<p>With BromBone you don&#x27;t have to either. You just have to make a small change to your .htaccess file.<p>BromBone crawls your websites, runs your javascript in a web browser, and saves the rendered page. When Google crawls your site, proxy the requests to BromBone, and we&#x27;ll send the rendered snapshot to Google.<p>Smile. Your javascript driven website is on Google.
======
tech-dragon
You have me wondering about the feasibility of offering this as a CloudFlare
app.

If I was able to throw up a little ajax/client side JS driven app at a domain,
throw CloudFlare over it (like I usually do for most smaller domains I own)
and then have it all happen without even needing to change that .htaccess
file, then your onto something even better.

I also think the pricing model may need some work. Per page seems like it
could cut you off from some potential customers. For instance, I'm already
thinking of a blog engine I wanted to try out more thoroughly by making it the
one i use on my new blog but wasnt prepared to loose "google-ability". I Would
like to be able to use a service like this but when I do my mental math it
doesn't add up. I wanted to use a blog engine that will produce a large number
of pages by pulling in third party service activity into my site. So its not
100% what I want, but damn its tempting.

If your pricing it to cover costs, might I suggest looking to optimize your
stack to bring down those costs down?. If you can cut the price and make it
more of a 'yeah for $X its easier than doing it myself' you'll get more
takers. Right now it feels that your current lowest price per month is too
high for a lot of devs to decide the $ is a throwaway expense to save them
time. Id hazard, below $25, and probably with an order of magnitude more
pages.

All that said... I may still try it out. It looks interesting.

~~~
chaddeshon
Thanks for your suggestion about CloudFlare. I am looking into it.

------
tocomment
Perhaps you could use crawl frequency in your pricing. For example daily (or
even hourly) crawls for the top tier plan.

Most people probably don't have that many pages so I'm thinking the existing
pricing tiers might not work well?

And maybe offer a free plan that only crawls your site 3x per year or
something really slow like that.

~~~
chaddeshon
I almost did that. Ultimately I decided to take an "it just works" philosophy
to crawl frequency (unless you need realtime).

Would you segment the number of pages differently? I think some sites with a
lot of user generated content might actually have a ton of pages. But it is
hard to get real numbers.

~~~
tocomment
Maybe Patio11 will chime in with some ideas for your pricing plans? Everything
I know I got from reading his comments/blogs.

(I think he searches for mentions of his username, so the bat signal is
officially turned on :-)

~~~
patio11
They're undepriced, but I think the first problem is going to be going from 0
users to 10 users (bang down doors) rather than extracting the maximum proper
value from those first ten users. The optimal pricing strategy will still lead
to him having 0 customers until he figures out who needs this and how to reach
them repeatedly.

~~~
chaddeshon
You're right. Getting these first ten is _SO_ hard.

After I have ten customers, I will have much more information to base my
prices and differentiators on.

------
chaddeshon
Clickable: [http://www.brombone.com](http://www.brombone.com)

------
tocomment
Do you think there would be any demand for a separate service that makes it
easy to screen scrape javascript heavy sites? (might be legal issues there
though?)

(Sorry for all the comments, this concept intrigues me)

~~~
chaddeshon
I don't know. That was brought up in my previous posting, and I've had some
people email me about it. It's hard for me to tell. I'm just not in that
community enough to understand their needs.

I'm afraid that every scraping project is just enough different that it would
be hard to write something that offered value to a lot of people.

~~~
tocomment
That's probably a good point. I guess if someone wants to screenscape a
dynamic JS heavy page they'd probably want to interact with the page.

------
tocomment
Marketing idea: How about you offer a free version but insert a link back to
brombone in the crawable/rendered version?

As long as you were very upfront about it, it might not be too offensive.

~~~
swah
Wouldn't that blacklist his own site?

~~~
tocomment
Why would it? I guess I don't know a lot about SEO.

~~~
wikwocket
Some people think that embedding a link to your site in a web product/plugin
you distribute can lead to bad things, especially if it is not a nofollow link
and your plugin is used by nefarious sites.
[http://www.jitbit.com/news/184-how-google-authorship-and-
pen...](http://www.jitbit.com/news/184-how-google-authorship-and-penguin-have-
killed-our-traffic/) has a bit about the Penguin algorithm and this effect.

------
tocomment
That sounds really useful! Will google penalize for this though?

~~~
chaddeshon
This is actually the method that Google recommends for these AJAX powered
websites. [https://developers.google.com/webmasters/ajax-
crawling/](https://developers.google.com/webmasters/ajax-crawling/)

I am just letting people out source the hard work. Since the rendered page
will be proxied from BromBone, there will be no way to tell that the same it
rendered the file instead of the normal webserver.

~~~
wikwocket
That's very useful info; I had no idea. I thought the Google crawlers were
basically headless Chrome bots so I wouldn't have expected this.

Do you mention this on your site? Seems like a good endorsement, "This is the
correct way to get indexed, as per Google!"

~~~
chaddeshon
I have a section about that. I hope people will read it.

