for ctx.Err() == nil {
select {
case <-ctx.Done():
return nil
case thing := <-thingCh:
// Process thing...
case <-time.After(5*time.Second):
return errors.New("timeout")
}
}
The extra check for ctx.Err before the select statement easily resolves the author's issue.
It really doesn't though. It handles the case where the context might have expired or be cancelled, but there's still a race when entering the select between the ctx.Done() and reading from thingCh. You may end up processing one additional unit of work. In situations where the exit condition is channel-based, this won't work.
Additionally, this would only work if you had one predominant condition and that condition was context-based. If you have multiple ordered conditions upon which you want to exit, I can't think of how you'd express that as a range.
I'm not sure what you mean. There's always going to be a race condition between ctx.Done and thingCh, just depending on whether there's data available. This race condition is unavoidable.
I guess you're thinking of "what if thingCh and ctx.Done activate simultaneously?"
There's no real difference between happening simultaneously and happening one after another.
As for your other point, you can just write code like
select {
case x := <-conditionA:
return x
default:
}
select {
case x := <-conditionB:
return x
default:
}
...
Also, this isn't semantically correct. In order to ensure that `conditionaA` is _always_ preferred over `conditionB`, you must also check if `conditionA` has received a value inside of `conditionB`:
select {
case a := <-conditionA:
return a
default:
}
select {
case b := <-conditionB:
case a := <-conditionA:
return a
default:
return b
default:
}
It would be easier to discuss with a more concrete example. If I ever had to write code like the above I would reconsider the design and try to come up with something simpler.
Honestly the whole first select seems redundant; any code that relies on this is broken as there's no other synchronization points to hang onto. You simply can't pretend the clock on the wall has anything to with the values transiting the program unless you introduce an actual synchronization point.
But OK, maybe you do have some strange performance case where this matters? In that case the whole thing could be more succinctly solved by looping on `for ctx.Err() == nil` instead of infinitely. Exactly as suggested at the start of the thread. (This would also likely be faster unless the context is under massive contention.)
It also leaks the timer until it fires if the context cancels, which seems like it would be more of a practical performance problem than any overhead to the additional select.
The context was introduced by the commenter. The original post does not use contexts. In general, there's a pretty common set of patterns in which multiple goroutines are writing data to different channels, and you need to ensure the data from those channels are processed with some level of priority.
Two channels is a poor way to handle priority. If data comes in on the low priority channel just before the high priority channel, you would still be blocked waiting for the low priority task to complete.
In a case like this, maybe just run two different consumer routines for the two channels, then neither would be blocked waiting on the other.
Context cancellation propagates (potentially) asynchronously anyway, so if you're relying on something canceling your context and that immediately appearing you already have a bug.
I've written `select { ..., default: }` enough times I also wish it had shorthand syntax - sometimes it's even clearer to range one "primary" channel and lead the code block with that check - but I cannot think of a case where relying on a deterministic select would not have led to a bug.
I think you could do which I'd argue is more idiomatic (
for {
if _, ok := <- doneCh; ok {
break
}
select {
case thing := <-thingCh:
// ... long-running operation
case <-time.After(5*time.Second):
return fmt.Errorf("timeout")
}
}
edit: nvm, your break would be blocked until one of the other channels produced a value. you'd need to check for the doneCh redundantly again in the select.
The author of this paper alerted Google on June 11, 7:35 AM EST, less than 6 hours ago. While we recognize this is a rapidly-evolving space, a few hours is not in line with responsible disclosure[1] timelines.
While we're still preparing a proper response to the submitter, the paper makes an invalid assumption that RPI rotation and BLE address rotation are out-of-step and overlap. The BLE and RPI changes are synced; the MAC address is always rotated with the RPI/packet is rotated. We're still investigating our implementation to verify, but we do not believe this to be a vulnerability. I will reply to this thread should our investigation find anything.
> Can anybody speak to the expertise of the authors on security?
I think a cursory LinkedIn or social media search for any of the title authors or chapter authors will demonstrate their credentials. There were many people involved in this book, all of whom carry the necessary credentials and experience.
> Personal questions for the responder:
Guidelines help us scale, but at the end of the day, some services are unique and require additional review. I recommend reading the bits on threat modeling for more information.
Generic credentials and experience provide very little information to me on their expertise. The CSO of JP Morgan, James Cummings, was highly experienced and credentialed when JP Morgan was breached in 2014 in one of the largest data breaches in history. The CSO of Equifax, Susan Mauldin, was highly experienced when Equifax was breached. The head of security for Windows at Microsoft is probably highly credentialed and experienced, but we all make fun of the insecurity of Windows. This is why I am interested in the specific projects they worked on and how they stack up. It is much harder to game the system if there is concrete, auditable evidence backing their expertise.
Yes, guidelines are not the end-all-be-all and you can never be sure, but when a civil engineer approves a bridge, they assert that they are confident that human lives can be trusted to the bridge (in certain configurations). They can do this with reasonable confidence because they have seen systems that have stood the test of time that prove out the techniques that they are applying. That is what I am interested in, do you/they have that level of confidence? What justifies that confidence? What systems prove out the techniques that were used? Did any techniques they invent stand the test of time (this provides evidence they can invent new techniques)?
The first book is a solid overview of how Google does SRE and outlining each of the various concepts (error budgets, blameless culture, etc..). The second is more of a practical guide on deploying SRE into an organisation, a lessons learned type of book.
(I work for Google but not SRE, just enjoyed reading the books)
Great question. As an O'Reilly author myself, I can tell you that we have no control over the animal selected. There's a fun animal selection process, but the publisher's decide.
Disclaimer - I work for Google and worked on this book.
O'Reilly provides their editorial capabilities. Google provides content. Neither Google, nor O'Reilly benefits from the book directly. Marketing. Future gains for both.
Hey everyone - Seth from Google here. Thank you for all the positive comments about the book. I'll be around to answer any questions you might have. As noted, the book can be downloaded for free in digital formats.
I really liked that there were HTML versions of the previous two books. Any chance that'll be up for this one?
A bit far-fetched but: Have you (or anyone else at Google) looked at Amazon Builder's Library [0] and/or various re:Invent / re:Inforce talks from 2018/19 [1][2] that focus on similar topics as in this book and other SRE books? If so, what are some ideas (infrastructure, blast radius, incident management, resilience, recovery, deployment strategies, crisis management, disaster planning, aftermath etc) you folks think that contrast / complement Google's approach to building hyperscale systems?
Download the epub version and on the Linux command-line execute `tar -zxvf srs-epub.epub` then cd into unpacked `OEBPS/` folder and there's your HTML files. Not exactly what you are looking for, but you can browse the content in a web browser.
I'll flag the HTML pages with the team. I'm honestly not sure. I know we're able to offer epub and mobi this time, something not previously possible with the other books.
I haven't looked at those other resources, but I'll ask if others have.
I actually wrote a script to convert the HTML version of the SRE book to epub, and then to mobi; which I then used to read a few chapters on kindle. It was far from perfect, but did the job.
No questions, just THANK YOU for this effort and for making it public.
We use it here to help expand people's minds, shifting their thinking from just writing applications to designing for large-scale HA systems, with all the fun pitfalls that lurk in a cloud.
There are multiple questions about what this book is about, who it's for and what might be relevant for me. We recommend going through the Preface to get answers to these questions. Copy/pasting a few paragraphs:
"In this book we talk generally about systems, which is a conceptual way of thinking about the groups of components that cooperate to perform some function.
We wanted to write a book that focuses on integrating security and reliability directly into the software and system lifecycle, both to highlight technologies and practices that protect systems and keep them reliable, and to illustrate how those practices
interact with each other.
We’d like to explicitly acknowledge that some of the strategies this book recommends require infrastructure support that simply may not exist where you’re currently working.
Because security and reliability are everyone’s responsibility, we’re targeting a broad
audience: people who design, implement, and maintain systems. We’re challenging the dividing lines between the traditional professional roles of developers, architects, SREs, systems administrators, and security engineers.
Building and adopting the widespread best practices we recommend in this book requires a culture that is supportive of such change. We feel it is essential that you address the culture of your organization in parallel with the technology choices you
make to focus on both security and reliability, so that any adjustments you make are persistent and resilient.
We recommend you start with Chapters 1 and 2, and then read the chapters that most interest you. Most chapters begin with a boxed preface or executive summary
that outlines the following:
• The problem statement
• When in the software development lifecycle you should apply these principles and practices
• The intersections of and/or tradeoffs between reliability and security to consider
Within each chapter, topics are generally ordered from the most fundamental to the most sophisticated. We also call out deep dives and specialized subjects with an alligator icon."
Actually the epub is so badly formatted, that Google Play Books does not even process it and fails. When i run it through epubchecker/Calibre, it shows 215 errors. Probably something you want to look at.
Particularly the "Building Secure and Reliable Systems" is targeted to software engineers. Copy/paste from the preface: "Because security and reliability are everyone’s responsibility, we’re targeting a broad audience: people who design, implement, and maintain systems. We’re challenging the dividing lines between the traditional professional roles of developers, architects, SREs, systems administrators, and security engineers. While we’ll dive deeply into some subjects that might be more relevant to experienced engineers, we invite you—the reader — to try on different hats as you move through the chapters, imagining yourself in roles you (currently) don’t have and thinking about how you could improve your systems."
This book should be suitable for software engineers without security background. There are some sections that might require some knowledge but they are explicitly marked as Deep Dive.
Unrelated to the book, but wanted to drop a thank you, Seth. You're one of the best speakers I've had the pleasure of seeing (HashiConf) and are a great example to me of what to aim for in public tech talks.
I’m on mobile safari and the ePub and mobi files open as text. This means I can’t export them to Apple Books or the iOS kindle app. Could you please trigger a download instead if possible ?
Thanks for the feedback. This is a known issue that another user flagged this morning. The team is pursuing a fix. The content-type on the file is incorrect :/.
In the meantime, you can open it in a browser and email it to yourself. Not ideal, but a workaround.
Thanks for putting together this in a book, I feel like I've written this book multiple times for compliance and explained the concepts but sadly that doesn't cross employment barriers. Also annoyed that I didn't make the time to write a lesser variant of this book first. I just hope this book's PDF name doesn't make my new title 'SRS' on the resume going forward.
