Several comments in this thread mention visualization. These data can be easily visualized with the Unidata IDV [1] (same ppl that make THREDDS). For those interested, go to the IDV dashboard, "Data Choosers" tab and enter
for the catalog. At that point you can browse the dataset, add the data as a data source, subset it (serverside), and visualize it. I and others have made a bunch of videos of how to use the IDV[2].
3. I was then able to add the data source via URL and saw two options in "Field Selector": "Image Collection" and "Omni Control".
I've tried selecting either field and clicking "Create Display", but nothing ever appears in the "Displays" tab. Would welcome further advice or more specific reference to directions among the 16 videos in your YouTube playlist. Thank you!
This may be related to some of the errors[1] that seem to occur on startup of IDV.
I found that downloading IDV at https://www.unidata.ucar.edu/downloads/idv/current/index.jsp and not using Webstart resolved the startup errors I reported above and now I see the Map View. However, I'm still not able to get any data to render on the map. Would love help still!
@bbuchalter Thanks for registering and downloading. We like it when our users register because we have to report usage metrics back to our NSF sponsors. Are you able to load the catalog (the .xml not .html suffix for the catalog URI) in the IDV dashboard, Data Choosers tab, and see the NCML files available at that resource?
@julienchastang thanks for your reply. I don't believe I'm able to see the NCML files. I've put some screenshots together here in sequence: http://imgur.com/a/4oI3q#0
1. Add the catalog.xml in the Data Choosers
2. Select Image Collection in the Field Selector and click Create Display
3. The error I recieve when I click create display: "Unknown XML root:catalog"
4. The subsequent Image Collection screen which I'm not sure how to use.
Thank you again for continuing to engage with me on this.
@bbuchalter I know you eventually downloaded the Desktop rather than Webstart version of the IDV and got that to work (this is the preferable option anyway, Webstart can be flaky), but just for the record, I believe you have to update Java on your host by going to java.com.
I just want a single image that tells me the expected better/worse degree of change in suitability for agriculture and habitation. Maybe red would mean a lot worse than today, and green would mean a lot better. An absolute image (of today's climate) would help make sense of the deltas also.
http://climateinternational.org has a set of visualizations for daily maximum temperature created from this dataset. Hopefully more visualizations will be added soon.
The entire dataset is 12TB because it contains _daily_ global 25x25km maps for all years between 1950-2099 probably for a bunch of different variables.
You don't need to download the entire thing to get the gist of it, although it probably would have helped if they also delivered a digest version with for example yearly or 5-yearly averages.
Edit: There is also a THREDDS data catalog present which allows you to extract slices or subsets of the entire data. Other than that it's missing a nice visualization tool I'd say its a pretty reasonable way of making such a huge amount of data available.
Not based on this dataset, but I gather most models already require an assumption of powerful carbon capture tech on a big scale to result in an outcome other than disaster. i.e. reduction alone is no longer sufficient in itself to prevent serious consequences.
I believe humanity will survive, but at a significant cost to lifestyle (at least from the point of view of this American). I'll miss the places that are beaches right now being beaches, and I'm not looking forward to the economic disruption as food sources change.
We're an advanced enough society that in a nightmare scenario, we could pretty much weather several generations in a cave under a tightly-controlled resource regiment. But not seven billion of us, and I don't look forward to the culling that would happen as a result of massive climate upheaval.
Would be great to see a nice visualisation tool built on top of OpenStreetMaps or something like that which allows you to see the predicted temperatures for a given location though time. Sadly this is way over my capabilities, also a 11 TB dataset is not easy to work with :(
I'm also interested in seeing the possibility of seeing projections for my area, but yes, there doesn't seem to be a good way of sifting through the data.
Would you be open to Kickstarting this project if the resulting repo was public/open source on Github?
EDIT: I should be clear. I would not be writing the code. I'd do the legwork to find someone (and pay them) who is familiar working with OSM to process the data, create the dataset, and build a javascript frontend to visualize it. The result would be open source. I would accept no funds whatsoever for this.
I don't have the time to do the project, but that doesn't mean I can find someone who does, and get them paid to do it and make the results freely available.
In fact, I don't think there's even a way to get historic temperature by day/location anywhere. That would be a useful thing to add, just the ability to down small subsets of the data.
If the earth is 509 million square kilometers and the granularity of the data is 25sq km this means we have about 20.3m data points. It implies then on average each point is 540mb. The dataset seems to be broken down by year rather than location though, a UX issue I'd say...
WRT breaking down by year, you could save some download time that way.
Lets say you express the predicted error in the data not as a percentage of temperature but as a time along the trendline. That would imply beyond a certain estimated error horizon, it doesn't really matter which year you download as long as you get the decade vaguely correct, once the temperature error figure exceeds a couple years of travel along the trend lines.
By example lets say today, here, 2080 CE, is 35C +/- 2C (to make the math easy) but the trend line is 0.1C/yr (unrealistic, but make the math easy) then +/- 2C is the same thing as saying +/- 20 years along the trendline, so I don't really need the 2080 data set I can do "roughly as well" with any data set downloaded from 2070 to 2090.
Yeah yeah I know calculus and the derivative is likely not constant or even linear and this is a really simplified way to look at statistics, but the general plan holds as a way to cut back on downloaded data required. Based on REAL statistical analysis you could come up with a formula that says you'll only increase the error bars 10% wider at year X if you skip every Y years of data. Where I'm guessing Y might exceed a decade at the extreme future years of the run. That could save an enormous amount of bandwidth without significantly impacting a visualization or analysis.
"Someone else" has to download it all, run the stat analysis, then tell us all, obviously it doesn't scale for all of us to download it all, run our own stat analysis, then go back in time and only download every Y-th year at or beyond year X. So thanks in advance, "Someone else"!
Its frustrating that they are not publishing the data in such chunks. Its good that they make every year available, but for analysis/reporting, you could reasonably just care about years 2030, 2050, 2100.
1 year of data would mean 120GB files or so (assuming 100 years total)
Breaking it down by lattitude would mean 800MB lattide-year average data sets. Breaking it down by longitude would be very helpful too, and not much reason to not make that breakdown in addition to the lattitude one, which means 5mb data chunks.
Its almost as though they are being intentionally uncooperative by dumping 12TB.
Hopefully, some group makes a web interface that allows downloads in these manageable chunk sizes.
NASA has released detailed global climate change projections. If you would like to know what these projections are please drink from this data firehose.
I might be wrong, but I believe the THREDDS server the article points to (by way of https://cds.nccs.nasa.gov/nex-gddp/) supports OPeNDAP, which means you can pull subsets of the data, i.e. if you've got a 40GB file, you could slice a tiny portion of that off for downloading.
I work with the THREDDS team at Unidata. Though I still need to study this particular dataset, in general yes, the THREDDS server can do serverside subsetting of the data so that reasonably sized data is shipped over the wire.
Those who deal with climate data are used to huge data sizes and are usually equipped with fast internet connections. You can also work with the data within AWS. The files come split in logical 750MB parts. There are different aspects of the data which come separately.
One could make the visualisation app work for a reduced data set and then constantly add to it whilst downloading. It would also make development easier. So you start with 2-3 years, make the app, make some scripts to automatically download and process what you need to process, and then add the other 147-148 years. This means... downloading only 164 - 245 GB of data... that'll keep me and my 10mbps broadband busy for more than 2 days :)
All of the ISPs where I live have a 150GB per month cap with $10 for every 50GB over, so this also means it would cost me an additional $2,500 in charges in addition to the download time. I don't think I'll be getting this anytime soon.
That would help but the cost of storage for 11 TB and the up time to download would be extremely expensive. I haven't seen an option that works for less than around $200 to rent and setup.
It seems like if these large public datasets continue to come on line, there will need to be some sort of semi-cooperative distributed data store to make them truly "accessible." Or the data provider will need to provide an access/query API, rather than just a big tank that you can copy if you dare.
I sure hope so because I am very very curious. Surely there are a lot of researchers that will love to jump into this data and manipulate it somehow. It's a good time to be studying climate change now... your dissertation is right here :D
Indeed. The image at the top of the page (presumably the worst case scenario) has no legend. I'm guessing that the red is desert, but what counts as temperate? The South America blue or the South Africa yellow? Or are the colors a delta from the current temperatures?
It's amazing how easy it is to turn a 12TB deluge of pristine scientific data into something completely meaningless.
My guess is that it shows the average temperature rather than actual climate. So red would be high temperature, and blue cold. I don't think that it refers to precipitation or anything like that. Or it could be a stock photo...
You'd think some spots on the globe would get colder or rainer, no? Also this map makes it appear Antarctica will get colder? Or maybe that map is a stock image?
Assuming the data is accurate and hasn't been 'adjusted' or 'corrected.' The whole climate business is a corrupt mess. Raw data over the years has been 'fixed' and policy is being made from 'models.' Yet what non partisan organization checks the models? The IPCC certainly doesn't. It's all a scam. We had global cooling fears in the 1970s. Then warming. Now "climate change." Yet industrial carbon output has risen exponentially and consistently since the industrial revolution yet temperatures haven't, thus calling into question that increased carbon dioxide raises temperatures. Such a disgusting mess.
Nope, the two terms have been used in the scientific community for decades. Climate change has always been the more popular term in numerical analysis of the scientific literature.
>Yet industrial carbon output has risen exponentially and consistently since the industrial revolution yet temperatures haven't
This is factually inaccurate according to several datasets published by independent scientific organizations around the world. How many do you want? Let's start with NASA, Japan, and satellite data:
"The most recent climate model simulations used in the AR5 indicate that the warming stagnation since 1998 is no longer consistent with model projections even at the 2% confidence level."
Kudos to them for publishing these, though. If they are rotten, it ought to be easy to tell.
What would be very interesting is for ESR to publish his own predictions, as he feigns expertise.
Or, hell, any global warming denialist to publish predictions at all, or build any model at all beyond their day jobs of quote mining and writing clever PR releases that prey on the public's scientific illiteracy and fossil fuel barons' desire to violate other people's lives and properties with untrammeled carbon pollution.
One can be a critic of predictive climate models without being a Denialist. I am. I strongly support both technological (nuclear and solar/storage) and political (carbon tax/tariff) approaches to ACC, but as a computational physicist I can't deny that climate models being predictively accurate--especially at the scale of these predictions--would be close to miraculous.
This is not a political position, it is an expert's evaluation after having looked at the code and documentation for some of the better models. You simply can't parameterize away as much of the physics as they do, or impose conservation conditions by hand the way they do, and expect a long-term integration to produce anything but the crudest approximation to reality.
System-wide averages are likely accurate in terms of scale. That is, global heat content is increasing at the order of 1 W/m2, probably not 0.1E-2 W/m2 and certainly not 10 W/m2. This is important, because 1 W/m2 is consistent with observtions and likely problematic in terms of local climate. The global economy is heavily optimized to the current climate, and even relatively modest changes would turn trillions of dollars of investment into malinvestment. This is a bad thing, if you care about global capitalism.
Climate scientists are not computational physicists. They have not spent most of their careers modelling systems that are ultimately subject to laboratory testing, and as such they have not seen their best laid plans gang aft agley.
I have long wondered how much of the hubris in climate prediction was coming from the policy level and how much was coming from the scientists themselves. This release suggests that it really is coming from the scientists, and that's unfortunate. Twenty years from now most of these predictions will prove to be false. That falsity will be in all directions: some will have temperatures or rainfalls far too low, some far too high. And the enemies of science will use that to further attack us.
Unfortunately, when I stand up and say this, I get attacked as an enemy of science by people who think they love science, but who are actually drawn to it simply as a useful stick with which to beat their political opponents.
If one's argument is that we don't know enough about what really drives our climate to make these types of predictions, why would one then turn around and make a prediction?
Also, using the phrase "global warming denialist" distracts from your point. When some scientists thought neutrinos traveled faster than light, they weren't called "light speed denialists". Have some respect for alternate scientific opinions.
The problem with GW predictions is that most of them are wrong and the ones that are correct aren't correct for the right reasons.
As an ardent defender of science, I tend to agree. There's always been a nagging intuition I've had about climate change that to this day I haven't resolved.
It comes down to the evolutionary history of Earth throughout life's 4 billion year tenure on this planet. The climate has gone through EXTREME changes [1][2], including the Chicxulub impact, which radically impacted the atmosphere's composition and life on this planet. Yet, the biosphere has always adapted and life continued.
Assuming that mankind is destroying the biosphere at an unprecedented rate (we are), and burning hydrocarbons at an even faster rate (we are), are these two factors enough to turn the earth into a Mars-style wasteland, completely devoid of all life?
It just doesn't seem plausible to me on an intuitive level. We can use linear regression models of the atmosphere and show temperatures rising until life becomes impossible, but that is not how complex biospheres work.
In any case, climate science is definitely a worthy scientific discipline to study, but it is becoming akin to economics in the prediction department. As in, the principles are sound, but the predictions of the experts are laughably wrong on a regular basis.
Unfortunately, this is all due to Al Gore and his "moment of genius" which instantly politicized the scientific process, thereby destroying any potential for real scientific progress through the feedback loop of empirical testing and model formation. If you disagreed with Al Gore's version of climate change, Al Gore's version of climate models, and Al Gore's version of climate science, you suddenly became a heretic "climate change denier".
It happened earlier than that, or at least according to a science journalist friend who interviewed MIT Professor Richard Lindzen in 1990 plus or minus a year. He quoted Lindzen saying that he had thought scientists were interested in the truth....
I'm old enough to remember when the Scientific Consensus was that we were causing global cooling. Amusingly enough, one of those guys became one of the most notorious global warming "scientist".
ESR is a global warming denialist. It's possible for people to publish hypotheses that don't support or that contradict climate change, but it's frankly playing it a bit too cute for you claim what he's doing at all approximates scientific discourse.
Like, for real. He more or less explicitly lied when reviewing the stolen Climategate source code, very deliberately and consciously presenting commented out code in a way to deceive his audience and provide talking points for right wing media. And when called out on it, he refused to take it down or even acknowledge the critique.
Unless you also think it's mean to call Rush Limbaugh a global warming denialist, calling ESR out on this is totally fair game. He's a hack. I wouldn't say the same thing of, say, Lindzen, but ESR is very committed to motivated, destroy-the-liberals reasoning, for the good of the Party.
He can certainly produce good content when he wants to. It's just when you give him the chance, his ideological blinders make him go wild. Forcing him to make concrete predictions and disciplining him with empiricism would move his content from dealing with fantasies in his head to the actual existing world.
It's unclear what you're even arguing for. Naive falsificationism? Even Popper didn't argue for that.
Science works by building theories and programmes of research. A single counterexample, contrary to what you seem to think, doesn't invalidate a theory--it's driven by multiple lines of evidence. And ESR's strategy of digging through comments of source code to quote out of context fragments doesn't even approach providing a single counterexample, particularly when he admits that he's in a quest to show that climate scientists are a conspiracy, part Gaian-religionists and part KGB psyops. Like, really.
> Noting that “buoy data have been proven to be more accurate than ship data,” the new study applies a new “bias correction” to address the difference between them.
..by adding a correction to the more accurate buoy data, generating a combined dataset that does not match trends in the satellite data, gold-standard terrestrial data, balloon data, or oceanic temperature data.
It also reduces the long-term temperature trend to the pre-1950 trend (i.e., pre-anthropogenic trend).
This is a paper that requires a significant degree of corroboration rather than simple endorsement because of its publication.
1950 is generally accepted by both AGW skeptics and proponents as a time before which human influence on climate was not discernable. That is, the warming trend before 1950 was natural, not human-caused.
AGW skeptics assert this places a burden of proof on AGW proponents to show that warming since 1950 is beyond what could naturally occur.
During the Climategate fiasco, Raymond's ability to read other peoples' source code (or at least his honesty about it) was called into question when he was caught quote-mining analysis software written by the CRU researchers, presenting a commented-out section of source code used for analyzing counterfactuals as evidence of deliberate data manipulation. When confronted with the fact that scientists as a general rule are scrupulously honest, Raymond claimed it was a case of an "error cascade," a concept that makes sense in computer science and other places where all data goes through a single potential failure point, but in areas where outside data and multiple lines of evidence are used for verification, doesn't entirely make sense. (He was curiously silent when all the researchers involved were exonerated of scientific misconduct.)
"My favorite part of the "many eyes" argument is how few bugs
were found by the two eyes of Eric (the originator of the
statement). All the many eyes are apparently attached to a
lot of hands that type lots of words about many eyes, and
never actually audit code." -Theo De Raadt http://marc.info/?l=openbsd-tech&m=129261032213320
Here are some citations showing the exact opposite. Believe it or not, scientists are humans and have biases, are occasionally bad actors, and sometimes just need to get that grant.
It looks like the origin of the use of the term "quote mining" is an article [1] on ScienceBlogs. I think your interpretation of the term is in agreement with it, in this case.
> Most of the environmental movement is composed of
innocent Gaianists, but not all of it. There’s a hard
core that’s sort of a zombie remnant of Soviet psyops.
I don't know about Australia, but that's certainly true of New Zealand. A large number of Communists and their hangers-on decamped for the Green movement with the fall of the Soviet Union.
This is coming from a perspective of "I know a little about stats", not "I know anything about climate science":
That doesn't feel like a particularly strong response to ESR's claim. Taken as a response to that claim, it feels kind of like: "our predictions were wildly inaccurate, but it's a complicated system with short-term and long-term factors and the short-term factors caused more warming just before we made our predictions and less just after, and that threw us off".
Which doesn't inspire confidence. If you can't predict the short-term factors, you need to widen your confidence intervals. I feel like at the very least, you should be making predictions like "conditional on these factors staying within these bounds, we expect this amount of warming". Then if that condition doesn't hold, you don't lose any bayes points.
(I recognize that the article wasn't written as a response to ESR, I'm not even sure it was a response to the same thing that ESR is talking about. But you offered it as a response, so that's how I'm evaluating it.)
Actually, what it says is that the results (which are themselves not raw data but a conclusion inferred from a wide mass of raw data) that ESR is saying weren't consistent with the IPCC projections have since, themselves, been reevaluated, adjusted to correct for biases introduced by changes in measuring methods, and corrected, and no longer present the problem he is pointing to.
The greatest changes in the new NOAA surface temperature analysis is to the ocean temperatures since 1998. This seems rather ironic, since this is the period where there is the greatest coverage of data with the highest quality of measurements – ARGO buoys and satellites don’t show a warming trend. Nevertheless, the NOAA team finds a substantial increase in the ocean surface temperature anomaly trend since 1998.
...
The global surface temperature datasets are clearly a moving target. So while I’m sure this latest analysis from NOAA will be regarded as politically useful for the Obama administration, I don’t regard it as a particularly useful contribution to our scientific understanding of what is going on.
------
The link above also includes more on some of the specifics of the new adjustments NOAA made.
What does Ted Cruz have to do with anything? Do you know anything about his positions? Wasn't Al Gore that said all of the polar ice caps would be melted by now? Wasn't it Gore that also predicted a dramatic increase in hurricanes? When did that happen? Cruz is Linus Pauling compared to Al Gore.
The point is that she shows her political biases by attacking Obama and defending Cruz. As far as Gore, I see you've ingested the appropriate Two Minutes Hate.
Ahh, yes, the famous Judith Curry, known by climate scientists as a quack who seems to be rather incapable of understanding the most basic notions of physics necessary to build (or refute, for that matter) any sort of argument in climate sciences.
Are these the same climate scientists that were accused of fabricating data by adjusting the adjustments in order to support their political desires? Those climate scientists? Because if anyone is having trouble with credibility right now it's the entire industry of grant-chasing climatologists. The IPCC even admired that climate policy was a means to redistribute world wealth. Does that not scare anyone? Has Al Gore become a billionaire because of a climate 'crisis' he helped invent? I don't know anything about this Curry woman but I do know that political ideology has a disproportionate influence in climate science because it's an ideal Trojan horse for Marxism. It's no accident that extreme leftists are the ones most passionate about carbon. Carbon represents freedom. It's that simple. Even the founder of Greenpeace admits that the whole global warming industry is nothing more than a political scam.
Seriously dude, go back to the conspiracy theories. Maybe go out, meet an actual climate scientist and read the actual scientific literature. You might learn a thing or two.
The science behind climate change has nothing to do with any personality or politics and everything to do with physics and atmospheric sciences. Or are you going to debate that the radiative forcing of an atmosphere with CO2 at 450ppm is magically less than one with 250ppm?
In your various comments you display such a startling misunderstanding and ignorance of how science itself works that it must be hard for anyone educated on these subjects to know where to begin.
The accusations you mention completely flopped and the "accused" were exonerated by half a dozen independent committees and investigations.
Climate scientists do not on the whole make a lot of money for their level of technical skill and training- if any climate scientist was "in it for grant money", they simply would switch to data analysis in the private industry or another field. Climate science is a diverse and huge field with many research areas that existed long before our awareness of climate change, and researchers' careers do not depend on the current climate change consensus- it's not like every university would simply fire their Earth Sciences department, or the NSF would stop funding Earth Science grants, if the climate was not warming. Additionally, you don't seem to understand how the grant process works- grants are awarded before results are obtained, chosen by committees of scientists, and a majority of climate science grants do not even write proposals concerned with future predictions. Finally, the irony is of course, the handful of denialist scientists are the only people who have been explicitly (and universally) linked to outside conflict of interest funding in their work.
Al Gore did not "help invent" any part of climate science, he merely brought popular attention to the issue.
Your failure boils down to the following: you have no perspective of the magnitude, operations, or workings of the scientific community. You have accused tens of thousands of scientists, safely employed at universities across the world, in countries with wildly different politics and governmental situations than the US, of conspiring together to produce dozens of scientific papers every week and an entire worldwide network of data collection, from satellites to buoys to promote a false theory, all of which gives them essentially nothing to gain.
Have you ever read a journal in the Earth Sciences? Can you even name one? If not: you haven't even begun to examine the scientific matters at hand. When you realize how wholly out of your field you are, it should become clear that your concerns are rooted in anti-intellectualism and rabid ideology, and nothing else.
Moreover, if the premise is wrong, the conclusion can only be false, that's basic fallacy.
Follow those models like they actually say anything is like giving a driver's license to a good racing gamer (NFS, GTA, etc.). It's just simulation, it's not real.
Weird how a "quack" manages to get her work into top-tier peer-reviewed journals, isn't it.
When one side of an argument descends to ad hominem as their first move, it is a sure sign their side has lost the argument.
Unfortunately, ACC is likely real and deserves a strong policy response. People like you, who admit to having no actual argument for their position, are actively damaging the cause of science and good policy.
Yeah, that one, versus the other 97% of other climate scientists who complain that her work is BS and that she has purposely misinterpreted the findings and conclusions of the IPCC reports.
Your idea that this is an ad hominem is intellectually dishonest. Curry has been criticized by the vast, vast majority of climate scientists for her outrageous and ignorant interpolation of other climate data.
Finally, there is nothing "weird" about people who are quacks publishing things on Nature. Nature is not the end-all of scientific publications, and it's not by any means a specialty publication in the field of climate science.
I'm not sure how discussing statistics is racist, particularly when later in the article he says:
"And that is actually a valuable hint about how to get beyond
racism. A black man with an IQ of 85 and a white man with an IQ of 85
are about equally likely to have the character traits of poor impulse
control and violent behavior associated with criminality — and
both are far more likely to have them than a white or black man with
an IQ of 110. If we could stop being afraid of IQ and face up to it,
that would give us an objective standard that would banish racism per
se. IQ matters so much more than skin color that if we started paying
serious attention to the former, we might be able to stop paying
attention to the latter."
Making discussing statistics "racist" is a disservice to disadvantaged communities.
It's racist to only quote statistics that support one's racist point of view.
And it is racist to be credulous of statistics that support one's racist point of view. Statistics come from actual data, so shortcomings in the data collection can produce shortcomings in the statistics.
For example the testing that goes into IQ is not without controversy. And crime statistics are directly dependent on the policing and prosecution of crime--they don't reflect the objective reality of crime, they reflect the outcomes of an imperfect system that detects and punishes crime.
There is ample and growing evidence that many such systems are significantly biased by racism. See: the recent Justice Department investigations of municipal police deparments like Ferguson, Cleveland, etc. So judging the criminality of a particular "race" by aggregate police statistics is not likely to be objective.
Finally, the concept of race itself is socially constructed, and therefore not objective. If a person has a white mother and a black father, are they black or white? What if one grandparent is Asian? Ultimately the attribution of race is a human social decision--a person self-identifies with a particular race, and/or others define their race based on how they look. Neither is scientifically objective the way that height or eye color is.
> It's racist to only quote statistics that support one's racist point of view.
> And it is racist to be credulous of statistics that support one's racist point of view.
Note that these are kind of circular. You've judged that the speaker's point of view is racist, and therefore his use of statistics is racist.
Also note that, to a first approximation, everybody only quotes statistics that support their own point of view, and everyone is credulous of statistics that support their own point of view.
It's easy to think of reasons why statistics would be biased in favor of showing IQ differences between races. It's also easy to think of reasons why statistics would be biased against that. In a world where most people don't believe in IQ differences between races - or even a world where people don't want to look like they believe in them - there's going to be publication bias against that direction. That's true even if there are no true IQ differences between races.
> Finally, the concept of race itself is socially constructed, and therefore not objective.
Imagine a world where there was a single gene which added ten IQ points to someone. (Which is definitely not the world we live in.) Imagine it was a mutation which arose in a single location a thousand years ago. It didn't become fixed in that population, and it's spread outside. But it's certainly more common in people who were born in that region, whose parents and grandparents were born there, than in people born on the other side of the world.
In this world, it seems to me that race would be just as much a social construct, and therefore not objective. But it would also be true that there would be a strong correlation between IQ and race.
> And crime statistics are directly dependent on the policing and prosecution of crime--they don't reflect the objective reality of crime, they reflect the outcomes of an imperfect system that detects and punishes crime.
That's true for drug use crimes and to a lesser extend drug selling crimes, which make up way too many of the crimes prosecuted. Similarly for other crimes where police have a large amount of discretion to decide whether or not to arrest someone.
However there are also big differences in crime rates by race for murder. It's very hard to dismiss this as the result of how murder is policed. Police generally have much less discretion in how to deal with murder.
I'd like to see some mention of the fact that blacks are incarcerated at MUCH higher rates for committing the same crimes as whites, which he never addresses (and he ignores it when people bring it up), yet that totally undermines his statistical argument.
His point is based on the rates of crime commission, not the rates of incarceration. The source cited by the article you link to says that "Yes, African Americans and Latinos disproportionately commit certain crimes", and so actually provides some support for ESR's argument. They do say that for drug use crimes, the crime rates are about the same for blacks and whites, but ESR was specifically talking about violent crime.
Also, they say that the differences in crime rates by race can be explained by socioeconomic disadvantages. But among the effects of socioeconomic disadvantages are poor early childhood nutrition and more likely exposure to toxic environments (hello, old buildings with lead paint!). These things tend to hold back cognitive development, which tends to result in lower IQ scores. So to deny a link between lower IQ on average and higher crime rate on average is in effect denying a link between socioeconomic disadvantages and higher crime rates.
Whether or not someone is being racist when writing about the link between IQ and crime depends on what they think is the causality behind the link between IQ and socioeconomic disadvantages, and the factors behind whichever one of those they think is the cause of the other. ESR may indeed be a racist, but you've not shown it, and the source behind your link could be reasonably cited to support the ESR statement you quoted.
We need to find some way to get some statistical sense into people at an early age, so as they grow up it becomes routine and automatic to recognize when you should care about group averages and when you should not.
In almost every situation where you are dealing with individuals, the fact that one individual comes from a group whose bell curve for some characteristic of interest is slightly displaced from the bell curve of the group that some other individual comes from is meaningless trivia when it comes to deciding which individual to choose for something that depends on that characteristic.
"His work and expertise in computer technology[9][10] has been all but overshadowed by his batshit insane wingnut tendencies in the wake of 9/11, as well as his increasingly wanky ego-gazing[11][12] as he acknowledges that people aren't so crazy about him anymore. "
>average 15 points of IQ lower than the general population?
Growing up in poverty, which many minorities do in this country, has been linked with lower IQ. Criminal populations are 8 IQ points below the mean. Whether there's a connection here is beyond me, but I really hate how people play up the hysterical political correctness card to censor these types of discussions. In many countries, raising people out of poverty lowers crime. There's definitely a connection here and in the US we're so PC, that we can't discuss these things, thus our continued high crime rates in many urban areas.
Calling everyone who disagrees with you a "racist" doesn't add to the conversation and only helps to make sure that we cannot discuss race issues without unneeded emotional drama.
OK, I just went and actually read Nelson's essay. It's not nearly as bad as I expected based on your writings. He's basically saying (badly) that because of racism blacks have to work harder than whites for any given level of reward and this changes where the optimum level of work is when you take into account work/life balance.
I don't know if he's right or not, but the idea is conceivably plausible. Suppose a white person can work 40 hours a week and make enough to live in a tier 1 neighborhood with a tier 1 lifestyle when it comes to physical goods, and suppose because blacks get lower pay for the same work, a black person would need 60 hours a week for that lifestyle.
If the black person only does 40 hours, like the white person, the black person can only afford a tier 2 neighborhood. Now what if the black person figures out that if he lived in a tier 3 neighborhood, he'd only need 30 hours a week?
If the gap between the quality of the neighborhood and goods affordable in tier 3 and tier 2 is relatively small, especially compared to the gap between tier 2 and tier 1, then going for tier 3 and trading away a small bit of housing and goods quality for an extra 10 hours to spend with family and friends or personal interests might easily be worth it to many.
For the white person, dropping from 40 to 30 hours would kick them from tier 1 to tier 2, which is a much bigger drop, and might not be worth gaining 10 hours of family and friend time.
So you end up in a situation where whites have a single peak in their work vs happiness curve at 40 hours. Blacks have two peaks: 30 hours for tier 3, or 60 hours for tier 1.
Using the term "lazy" for those who opt for tier 3 was a mistake. He doesn't seem to be trying to use it as a pejorative. I think he was thinking more along the lines of The Dude or Cosmo Kramer style laziness.
>"Now what if the black person figures out that if he lived in a tier 3 neighborhood, he'd only need 30 hours a week?"
Do you really think black people go through such mental calculations like that, when deciding what to spend their time on next? Do you, whatever race you are?
You and Russ are presuming that you have any idea what thoughts and mental calculus goes through a black person's mind when they decide when and how hard to work on what, and that you could even know or deduce it without asking, and that it's the same for all black people, and that there's a difference between how black and white people think, which rules and formula they use, and which numbers they plug into them.
Don't you see how condescending it is for a white man (or anyone) to lecture black people (or anyone) about how they think and why they behave, presuming to know better about the cause of their problems, explicitly calling them lazy as a class, acting like he's doing them a favor by justifying their laziness? That level of patronizing, smug, self righteous condescension directed at black people as a class is racist.
He's backward chaining from his racist foregone conclusion that "blacks are lazy" to a bunch of made-up speculation he pulled out of his ass about how "black people" think and what motivates their behavior, which he can't possibly know.
He purposefully led with as offensive a title as possible just to get attention, and then he tried to demonstrate how clever he is to be able to logically argue himself out of the huge hole he just dug with the click bait blog post title "Blacks are Lazy". And it's not just that his arguments are "badly written", it's also that his assumptions and logic are deeply flawed.
My theory: Russ was just inexpertly trying to ape Eric Raymond's favorite rhetorical tactic, and he failed miserably. But Eric was flattered, and stuck up for him.
“Personally, I wanted to fight this on principle,” Raymond said. “Russ resigned the presidency rather than get OSI into that fight, and the board quite properly respected his wishes in the matter. That sacrifice makes me angrier at the fools and thugs who pulled him down.”
The Dog Whistle Has Sounded: How the Right Talks About 'Thugs' Like Trayvon Martin
> Do you really think black people go through such mental calculations like that, when deciding what to spend their time on next? Do you, whatever race you are?
Of course not.
> You and Russ are presuming that you have any idea what thoughts and mental calculus goes through a black person's mind when they decide when and how hard to work on what, and that you could even know or deduce it without asking, and that it's the same for all black people, and that there's a difference between how black and white people think, which rules and formula they use, and which numbers they plug into them.
Nope. I'm presuming that black people make these decisions exactly the same way white people do, which is by observing the outcomes of the decisions of others similarly situated and picking a path similar to the paths taking by those whose outcomes they would like to also achieve.
They can see that blacks living in the tier 1 neighborhoods need to put in a lot more work than the whites living in those neighborhoods. They can see that putting in merely the same level of works that whites do results in at best tier 2. They can see that working less than that results in tier 3. They can see that their friends working less and settling for tier 3 are spending more time with family or personal interests. The can see that tier 2 is only a minor improvement physically over tier 3, and is a long way behind tier 2.
You're pointing me to the essay, which was written by Russ Nelson. I couldn't find any commentary by ESR on the essay, just on the reaction to the essay.
> My point is that ESR is not a trustworthy source of information.
Then the word to use for that is "untrustworthy". The word you've actually been using is "racist".
Rationalwiki is not a trustworthy source of information. That's not something I can provide a firm citation for, so I don't expect you to start ignoring the place, but personally I try not to believe anything it says without doing the research myself. (And on the subject of ESR, I try not to believe anything anyone says without doing the research myself.) I don't remember the details of ESR on climategate, and don't particularly care to look it up.[1]
I do remember this, however:
> ESR wrote a blog post suggesting that the Haitian people really did summon up the Voudon god Ogun to kill off all the white Frenchmen.
and it's not a fair description. ESR suggested that the Haitians really did attempt to summon Ogun. He suggested that the ritual was valuable for physchological reasons. He did not suggest that they succeeded in summoning Ogun.
[1] There's a failure mode that I sometimes fall into, where I don't care about an object-level discussion, but weigh in on the meta-level, because the meta-level is more interesting to me. Then I get sucked in to the object-level. I'm trying not to do that here.
So... you're saying the only possible opposition to his climate results is distraction via playing the race card, logical fallacies, and sophistry? So you're a strong supporter of his climate claims. Yet you were implying between the lines earlier that his climate claims are perhaps not so good. Which is it? Or is it just the anti-scientific "truth depends solely upon who says it"?
As a meta analysis I'm not sure dramatizing or personalizing a statistical analysis is supposed to improve anything anyway. So how deep does the sophistry go? Is there any reasoned response to the claims other than personal attacks against theoretical semi-libelous attacks against individuals based on unrelated political beliefs? Sadly, there probably is, so you're doing a disservice to that position. Are you are being paid to disrupt scientific reports about climate change by discrediting "supporters"? It is known there are people being paid to do that, although I'd expect hirelings to do a better job for their pieces of silver.
There recently was a new study that showed that the warming stagnation was not that much of a stagnation after all. Just measurement errors (and mixing up of data from different measurement methods). Interpretation differs, though - some say it is still a slower warming, just not as slow as was previously thought.
I know this because of the high pressure area that's currently sitting here and will take time to move.
Last weekend, however, it would have been practically impossible to tell whether it would rain here or not. There was a low coming in off the Atlantic, and if it had gone a little further North or South, it would have rained there instead of here.
Forecasters are nearly 100% accurate with what the weather will be. It's just the time and location of that weather that are tricky!
http://dataserver3.nccs.nasa.gov/thredds/catalog/bypass/NEX-...
for the catalog. At that point you can browse the dataset, add the data as a data source, subset it (serverside), and visualize it. I and others have made a bunch of videos of how to use the IDV[2].
[1] https://www.unidata.ucar.edu/software/IDV/
[2] http://goo.gl/n0Frpb