
The news done broke - webology
http://www.b-list.org/weblog/2010/nov/02/news-done-broke/
======
itsnotvalid
Next time they may ask the office to provide those data in PubSubhubbub
protocol so that active polling of data is no longer a requirement.

Just a side note:

    
    
      All screen-scraping techniques are vulnerable to the 
      “whoops, they redesigned” problem. This screen-scraping 
      technique was vulnerable to the “whoops, they redesigned and 
      we go live in a couple hours” problem.
    

These two sentences sound a little odd to me. Since the original page by James
has no comment box, hopefully he could pick this up here.

~~~
tkahn6
Why do these sentences seem odd seem odd?

Pulling data from HTML is highly dependent on its exact structure. Usually if
the page changes at all, you have to go back and figure out where everything
is again.

Probably by "we go live in a couple hours" he means that they didn't have
access to the page and its new format until a few hours before they had to
have it ready to send out.

~~~
itsnotvalid
It seems odd because like 90% of the words from the first sentence is repeated
by the second one. Seems not DRY enough for me. It's more on the style of text
than the problem, as I clearly agreed that by suggesting the data source to
adapt to publisher-subscriber model (or some xml/json format)

~~~
ubernostrum
Well, there are two things.

One is that my first look at the (new) HTML of the page I needed to scrape
came only a couple hours before polls closed, thus adding a bit of urgency.

The other is that, stylistically, I'm a fan of anaphora and similar tricks.

------
shubber
There was a period in time where I needed to do the same thing for sports
coverage - long story, but there's something amazingly gratifying about
ingesting huge quantities of data from semi-public sources in a time-critical
way.

~~~
sliverstorm
> there's something amazingly gratifying about ingesting huge quantities of
> data from semi-public sources in a time-critical way.

1) the feeling of being completely 'connected'

2) the feeling of being on the 'cutting edge'- knowing before 99% of the
population hears about it

