Hacker Newsnew | comments | show | ask | jobs | submit | login

I am quite new to Go, but I have been happily using this for a few months on a personal project. I really recommend looking at https://github.com/jmoiron/sqlx which is a library extending database/sql to simplify going from query->struct/slice and it also allows for named parameters and prepared statements.

#go-nuts on Freenode is a great place :-)

-----


While it may not look like it, in my opinion the uwsgi project desperately needs documentation help.

http://uwsgi-docs.readthedocs.org/en/latest/

There is just an endless list of features, and no clear red thread as to what you actually need to get going.

It's hard because it just supports everything for some reason.

-----


I see this in zsh 5.0.0 on Ubuntu, and on OSX as well! zsh 5.0.2 (x86_64-apple-darwin13.0)

-----


I'm not seeing this on zsh 4.3.17. Exactly what did you run to see this?

-----


They probably ran

  env x='() { :;}; echo vulnerable' bash -c "echo this is a test"
but failed to spot that the second command invokes bash not zsh.

My tests suggest that neither zsh 4.3.17 or 5.0.6 (the versions that ship with Debian stable & testing respectively) are vulnerable to this exploit - if you replace bash with zsh in the test oneliner then the code after the end of the function definition in the environment variable is not executed.

-----


That's it, I am an idiot today. (and more days probably)

-----


At the world cup finals in Hafjell, Norway last weekend, the top guys were hitting 82 kmh (51 mph~) on a open section speed trap. They aren't quite as fast in tighter sections, but still plenty fast.

Here is the helmet cam footage of Josh Bryceland that probably would have ended up winning if he had not overshot the last jump and broken his foot. https://www.youtube.com/watch?v=XBDpu4-pQhM The speedtrap is at about 2:30.

-----


Do not forget about suspension technology.

A mid-range fork today is so much better than a mid-range pogo-stick fork from the 90s.

Better forks & shocks as well as a slacker headtube-angle on most bikes today make todays bike a lot better to handle and ride, and inspire a lot more confidence than a bike from the 90s.

I do agree prices are high, but you do not have to have a $10k carbon bike to have fun or win races.

-----


Also, narrow/wide chain rings, clutch derailleurs and a far greater range of available gearing options has advanced the entire drivetrain compared to 5, 10 years ago...and lets not forget all the advances in disc brake technology over the years.

-----


Yeah, but going from eight to ten cogs on the rear isn't going to make that much of a difference to the rider.

-----


You are right, but it makes a bigger difference than you suspect. My bike has a 2x10 setup right now, and I am most likely converting that to 1x10 or paying up for a 1x11 setup in the future.

Not having to choose between 1-2 on the front and then the back gears simplifies things greatly. In addition you also save some weight, it's a much simpler mechanic with less maintenance and with a newer SRAM Type2 or Shimano Shadow+ Clutch derailleur and a narrow-wide front ring you pretty much can't drop a chain, which is great. The only exception would be if you ride incredibly rough terrain where you bounce a lot, but a very simple chainguide fixes that problem.

-----


Look for end of the season sales, or better yet, know someone that works for a supplier or store that can get you a decent discount. Mountain bikes are insanely expensive, but for local stores there mostly isn't a big profit margin (at least not here, Norway). The manufacturers themselves probably have a lot better profit margins, but I do not know enough to say anything about it. At this point, any "known-brand" mountain bike over or around $2000 new is a really solid bike that will work great and last a long time if taken care of properly. Above that you start to see carbon frames, higher end gruppos and mostly weight savings.

For example a bike with a Shimano SLX groupset is basically as good as one with the XT groupset (one step up), the XTR groupset is lighter and costs way more.

the Shimano SLX groupset is comparable to SRAM's X9 groupset in performance and price.

Getting into the sport I would go for a hardtail (no rear suspension) first. They are a lot cheaper, and you will get more bike for the money. Spend $2000 on a hardtail and you have a really great and light bike with a near top-end groupset, but if you spend $2000 on a full-suspension bike you will get less bike for the money, and a hardtail is good for beginners anyway. When you do not have rear suspension you have to learn how to keep your weight on the bike and how to pick the best lines down, and a full-suspension bike is a lot more forgiving that way. I can ride down stuff on my FS bike I'd never dream of doing on my hardtail.

-----


Would you recommend building something like this for a much smaller system? 10TiB or so maybe, I do not need that much, or do you think buying a NAS of some kind would be better?

I kind of want to set something like this up while spending the least amount of money. I am comfortable enough with Debian/Linux to do most things, but I have never managed anything like this. In the end I want to end up with somewhere relatively safe to store data pretty much in the same way you are, I just do not need 70TiB, and I have no experience with ZFS/hardware stuff/storage.

-----


By "something like this", do you mean ZFS? I am a HUGE fan of ZFS, and I do think that it's worth using in any situation where data integrity is a high priority.

As far as ZFS on Linux, it still has its wrinkles. I use it because, like you, I'm comfortable with Debian, and I didn't want to maintain a foreign system just for my data storage, and I still wanted to use the machine for other things too. (I actually started with zfs-fuse, before ZFS on Linux was an option.)

So, I don't know. If you just want a box to store stuff on, you might want to just look into FreeNAS, which is a FreeBSD distribution that makes it very easy to set up a storage appliance based on ZFS. FreeBSD's ZFS implementation is generally considered production-ready, so you avoid some ZFS on Linux wrinkles, too.

So, I'd recommend checking out the FreeNAS website, and maybe also http://www.reddit.com/r/datahoarder/ for ideas/other opinions. I do a lot of things in weird idiosyncratic ways, so I'm not sure I'd recommend anyone do it exactly how I have. :)

-----


If you're comfortable with Debian then you shouldn't have too many issues with FreeBSD as there is a lot of transferable knowledge between the two (FreeBSD even supports a lot of GNU flags which most other UNIXes don't).

Plus FreeBSD has a lot of good documentation (and the forums have proven a good resource in the past too) - so you're never going it alone (and obviously you have the usual mailing groups and IRC channels on Freenode).

While I do run quite a few Debian (amongst other Linux) I honestly find my FreeBSD server to be the most enjoyable / least painful platform to administrate. Obviously that's just personal preference, but I would definitely recommend trying FreeBSD to anyone considering ZFS.

-----


As far as I'm concerned, the most identifiable characteristic of Debian is the packaging system, dpkg/apt. I've used FreeBSD occasionally, and that's what I always end up missing about Debian. I did consider going with Nexenta or Debian GNU/kFreeBSD, but whatever, ZoL works well enough. :)

-----


FreeBSD 10 has switched to a new package manager, so it might be worth giving it another look next time you're bored and fancy trying something new.

I can understand your preference though. I'm not a fan of apt much personally, but pacman is one of the reasons I've stuck with ArchLinux over the years - despite it's faults :)

-----


I'll keep that in mind; I do sometimes find myself with some time to play with things. :)

-----


By 'something like this' I meant pretty much what you just said: Would you do it the same way (your own everything) if you needed a much smaller system, or would you go with something like FreeNAS, like you suggested? I am confident I c an get it working good either way, but I would rather not spend half my days having to tweak and worry about stuff working correctly. I understand that it will need maintenance and monitoring of course, but I would much rather be more of a end-user having a working system than being the sysadmin that has to fix it all the time. :-)

Thanks for the link, I will take a look there.

-----


Well, if you don't get a kick out of "tweaking and worrying", yes, I definitely recommend FreeNAS. Although I'm confident in my system now, it took a long time to get this way, and I could've saved hundreds of hours by just going with something like FreeNAS (had it existed); I stuck with it because I kinda enjoy doing things the hard way.

-----


I do kind of get a kick out of that, but at the same time I also just want a safe system for storing data. If I end up building something like this I will take a look at FreeNAS! Thanks!

-----


I have a similar setup with 12TB capacity. ext4 over mdadm RAID-6 w/ 2 spare drives. It's specifically setup such that any single failure (including SATA expansion card) can't bring down the pool. It's been stable for ~2 years, and it's really nice to have that much storage in the house.

You don't need ZFS for this, as cool as it is.

-----


ZFS still protects you from bitrot when compared to ext4 over mdraid. When you get to many terabytes of data, it's almost guaranteed that you're going to lose something to bitrot. In my case, my most recent scrub detected and repaired 1.58MB of bitrot. And in any given month, `zpool status` will show one or two checksum errors as having been corrected in real-time, as I was working with the corresponding files directly.

This is probably the number one thing that excites people about ZFS over any other solution, and it's something that isn't really easily implemented on a standard RAID + standard filesystem arrangement, since this sort of functionality depends on the filesystem knowing about the underlying disk arrangement.

"ZFS uses its end-to-end checksums to detect and correct silent data corruption. If a disk returns bad data transiently, ZFS will detect it and retry the read. If the disk is part of a mirror or RAID-Z group, ZFS will both detect and correct the error: it will use the checksum to determine which copy is correct, provide good data to the application, and repair the damaged copy."

https://blogs.oracle.com/bonwick/entry/zfs_end_to_end_data

-----


How is that any different from standard RAID? That's exactly the problem RAID was created to solve...

-----


ZFS & btrfs detect and fix silent corruption - where no errors are emitted from the hardware.

I think the pertinent question is: when the filesystem goes to read a 4K block, and one drive's copy of this block in the RAID-1 set is different to its counterpart 4K block on another disk, which one wins?

-----


I didn't specify RAID-1. RAID-5 or RAID-6 can reconstruct the correct value in a silent fail.

Honest question: how often to drives silently fail? Drives contain per-sector checksums these days, explicitly to prevent this problem.

-----


I don't know how often they fail, but I will say that the failures that I hope I never see again (or at least, I hope I never see again outside of a ZFS system) are not drive failures, but those involving intermittent disk controller or backplane faults. In comparison to the chaos I've seen this cause on NTFS systems, ZFS copes astonishingly well.

-----


A normal raid does not check the checksums on read. It only uses them after a device failure.

Also it may have copies of the data eg raid 1 but does not know which is correct if they differ.

-----


No, every default mdadm install performs a complete scrub on the first Sunday of the month. Every block of the array is read back and validated. For RAID modes with parity (e.g. RAID-5, RAID-6) it is able to detect and fix the offending disk when a silent error occurs. You can trigger such a scrub whenever you want (I run mine once a week).

-----


Scrubbing the entire raid volume is significantly different from scrubbing every piece of data as it gets written/read.

First, in between your monthly/weekly scrubs your disks/controllers will be silently corrupting data, possibly generating errors on multiple devices resulting in data loss depending on raid type. ZFS detects corruption much more quickly.

Second, your traditional raid recovery is to rewriting the entire device to fix a single block. Let's say you're using RAID5 and you're rewriting parity. You get another block error. Oops, now you've lost everything. Since disks have an uncorrectible block error rate of 1 in 10^15 bits, you only need a moderately sized array to almost guarantee data loss. ZFS rewrites corrupt data on the fly.

-----


Every time you read or write from a RAID volume, it does perform validation and write-back on error detection. I think your mental model of how linux software RAID works needs updating.

I'm not trying to argue that mdadm is better than ZFS, just that in this case they pretty much compare the same.

-----


If the _drive_ reports a read error it will. If there is silent data corruption it wont. You can test this by using dd to corrupt the underlying data on a drive.

-----


Hrm. I'm going to test that.

-----


That's interesting, I've been running a 6x3TB raidz2 for a year or so on wd reds and no bitrot so far, no checksum errors either, regular scrubs.

-----


Almost all of the bitrot I see is on the oldest vdevs, which at this point probably contain mostly only old snapshots that are almost never accessed. My oldest vdevs are... 4-5 years old.

-----


A more appropriate and less sensationalist title would be "Man on holiday uses his remote controlled quad-copter to search for missing man", or "Man with remote controlled helicopter got lucky, found missing man after just 20 minutes because he searched in the right place".

The 'drone' is not the news here, he could have been flying around in a helicopter, if he looked in the same place he would have found him.

-----


If by 'drone' you mean 'unmanned multirotor', it's actually quite an accurate topic. What's more, it provides more insight into why this is actually a big deal in modern society.

A 'drone' is a cheap and effective way to have a camera with a huge field of view operating in areas that humans and helicopters cannot, or just doing things that it would be prohibitively expensive to do otherwise.

You can fly a 'drone' over a river, where a human would be hard pressed to follow, and you can fly a drone amongst trees where helicopters could not fly. You can do both of these things cheaply, and in a far more scalable fashion. 'Drones' don't need roads or other such infrastructure, and can carry decent sized payloads practically anywhere.

The application of 'drones' has huge impact on a large number of areas of applications, and it would be silly not to recognise that.

Somewhat sensationalist it may be, but in my opinion the advent of these things is worth it.

-----


Exactly

Flying a drone is much easier and cheaper than flying a helicopter.

Sure, you can go ride a chopper to look for the person, it's about $500 an hour

Not to mention risk and hard to reach places.

-----


So in this case, that's $167.

-----


Add coming and going from its base to the search area, planning the search (flight plan), etc

It's not "hop in and fly" usually

-----


Yeah, the startup costs of helicopters are large. A Marine quartermaster once told me that it costs $N,000 just to turn a military helicopter on, because after you turn it off you have to do a bunch of maintenance tasks no matter what. And chopper technicians aren't cheap.

-----


I completely agree that mass surveillance is and will be a problem, and drones like that might very well be a part of the problem in the future.

But the 'drone' in this case did not do anything. It is a remote controlled vehicle, and it did not do any thinking, the operator did.

The title implies the drone was the key element in finding him, and not the person controlling the drone. I agree that drones will make surveillance easier, but until they fly autonomously without human control then it is a much smaller problem.

Using drones to find missing people is a good usage case though, and I like that.

-----


> The title implies the drone was the key element in finding him, and not the person controlling the drone.

In a sense, that is accurate. The person alone would likely have taken far longer to find the person, so the drone was a key (if not the key) element in finding the missing person so fast.

-----


Helicopters are really expensive, and require a lot of manpower.

>"Had we had not seen him then, the drone would have seen him a few minutes later since he was in the search area we were given to look at. If nothing else, the drone helped us cover a huge area in a short amount of time that would have taken many volunteers hours to search."

I think that the cheap and quick aspects of drones are really important here.

-----


This. Exactly this is why it's significant.

Even a small piston powered Robin helo is around $500 to operate per hour[1], and the smallest turbine powered ones are going to be around that figure just for the fuel: ~500 lb/h fuel consumption [2][3], JetA at > $6 / gallon [4].

So for one hour helo ops, you can purchase 2-3 parrot drones [5].

[1] http://www.aneclecticmind.com/2013/09/12/why-i-charge-545hou...

[2] http://www.turbokart.com/about_arrius.htm

[3] http://en.wikipedia.org/wiki/Eurocopter_Colibri

[4] http://www2.jetaviation.com/index.php/jet/bedford/fuel

[5] http://www.amazon.com/Parrot-AR-Drone-Quadricopter-Controlle...

-----


But you need to factor efficiency into that. If the top speed of the chopper is 100mph, versus 10mph for the drone. Then you can cover a much larger area quickly, furthermore a helicopter can stay up longer and has much greater range. So it's not enough to say a heli costs x/per hour therefore drones. Kind of like saying trash trucks are expensive, therefore we should replace them with wheel barrows.

From the article: We were asked to search a large area of farmland with the drone. I covered three-quarters of it using three batteries, and the last quarter was a little too far for me to get good first-person view reception

Landing to change batteries every couple of minutes, is a serious blow to efficiency.

-----


This is exactly what's wrong with most of today's news. Sensationalism is taking the focus away from what matters[1].

But I'm not sure if the drone is not the news here. I think a drone can be a cheap way to search from above. Instead of using a helicopter multiple drones could be used saving costs and covering a larger area.

[1] Read 'Avoid News'.

-----


people with the ability to immediately fly a drone > people with the ability to immediately fly a helicopter. The drone/rc/quadrocopter IS the news. The fact that one guy put a camera in the sky is news.

-----


I wonder why the story is sensationalized and hyped up so much in the first place, and who stands to gain from it?

-----


Currently there is a big argument between us UAS hobbyists and the fed over our rights to fly our model aircraft. Search and Rescue has been deemed by the FAA to be commercial use and disallowed unless the pilot and aircraft are certified which is nearly impossible and very expensive. Just recently a judge has ruled that the FAA does not have enforcement powers over this use but it continues to go after such use throwing it's weight around, possibly at the cost of human life.

-----


An alternative to this is RoboBrowse [1] which is also based on requests + BeautifulSoup4 and seems a lot more mature.

[1] http://robobrowser.readthedocs.org/en/latest/readme.html

-----


Note: this fails when you try to install using pip 1.1, the version that comes with Debian wheezy. If you have this problem, it's easy to work around it for your specific virtualenv, without changing the rest of the system:

- mkvirtualenv whatever

- pip install pip --upgrade

- pip install robobrowser

-----


Haha! If I'd known, I would have used that myself.

-----


Still, it proves the idea is a good one!

-----


I agree completely, you sound exactly like the kind of person I am!

I just finished my computer engineering bachelors degree a month ago, I am 26 years old. From all the people I know at school, only a small handful of them has ever done any kind of coding or any thing outside of the school curriculum (more than just reading a tutorial one night).

That is, for me at least, a frightening thought. You want to work has a developer but you do not spend any time outside work to learn?

Some of us are curious and try things out. I believe those that do are (mostly) the people that can be good developers. A situation that was particularly curious to me was when we had our first Programming 101 (Java) class the semester before Christmas, and then when the second class from around mid-February. In between that time, a huge amount of the class did not look at any Java, or do anything, or even try it out. So unsurprisingly, barely anyone remembered anything. There was just a complete lack of interest, and I can't figure out why they would choose this career path if they did not enjoy it and I do not understand how those people managed to pass classes, or how they are supposed to do good work later.

I have had a growing interest in computers since I was a child, and I think I have been coding since I was around 12-13 years old. During my time at university I have attempted to get people more interested in coding by suggesting they figure out some problem they want to solve, and then spend some time making something. I think I managed to get only a few of them to try anything, while the rest were questioning why they would want to do anything like that... I know quite a few of them have gotten jobs directly out of school, but I have no idea how.

Edit:

I do understand that not everyone has time to nerd out all the time, especially when you get older and have a family. I also like to relax and recharge, so I try to keep busy doing other things like exercising. Sometimes I just get so sick of work, school and everything that I can't seem to write any code at home for personal projects. It is nice to be able to burn of that unspent energy mountain biking for example. And sometimes that just seems to shake loose a solution to a problem I have been having or give me inspiration to create something new.

-----

More

Guidelines | FAQ | Support | API | Lists | Bookmarklet | DMCA | Y Combinator | Apply | Contact

Search: