ehhhh. Lots of hyper-conservative responses there (which is understandable, it's asking for legal advice). But assuming this is part of an RFP response, "embellishment" of answers is par for the course, and I understand the frustration of the director.
First, there's always a good chance nobody on the client side actually cares about this. RFPs tend to come with hundreds of questions, most of which are put in as requirements by completely detached departments to make them feel important, but don't actually matter. If something is important, it can be negotiated. For example, if the pen test is important, you can get the client to agree to sign the contract with a clause that says if they don't get pen tests results by X date the contract is void.
Second, well, what is a pen test, really? If you ever called one of your APIs to validate that your authorization or authentication code works, or you've validated that your AWS security groups are blocking external traffic to your database, congrats, you've performed a "pen test"! The client won't consider that sufficient, of course, but it's a justification for an answer on an RFP, at least. If they want more details, as they do in this case, they can always schedule a follow-up call or ask to see the pen test results document.
This situation reads like an engineer who needs a bit of business/sales experience more than anything. In their discussion with the client, you can start with "we've currently only done in-house penetration testing, but are looking to contract an external vendor to do penetration testing by X date." Assuming you've done any testing, this is a true statement, and the client can determine if that's sufficient.
The real solution to this is to get SOC2/ISO27001 certification, though, which makes a lot of these RFP headaches go away.
Lying is not an embellishment or puffery, it's a lie. Engaging a company for a 3 day pen test that's totally insufficient, that would be an embellishment.
It doesn't matter if the client doesn't really care. That's just a rationalization, plain and simple. No different in kind from, "well the bank teller doesn't really care if I rob the bank because it's not their money, and the money is insured anyway, so is it really even robbery?" Only different in degree (to be fair, an enormous degree; just to illustrate).
A pen test is a real thing. "What is a pen test really?" is another rationalization. There may be many flavors of pen test, but fabrications are fabrications. One of the most important part of pen tests is that they are external. It's like saying, "what is an audit really? We have accountants and they check our books for anomalies." Just doing your job as an engineer and looking for bugs is not a pen test. In the same way that being careful and rereading your own changes is not a code review.
This reads up me like an engineer committed to their work. I think they should be proud of themselves for not going along with this. I think the problem is that management isn't doing their job properly. They're cutting corners because they fucked up and didn't make sure a pen test happened or listen to their technical people. This is a strategic necessity for the company that would have been so easy to accomplish and should have been foreseen. They're trying to rule by dictate and it could destroy their career if this ends up in court. Even now, they could get some kind of rush job done - but no, they choose to endanger the company and the people in it instead.
Imagine being a lawyer or a paralegal and getting your hands on those emails in discovery. They didn't only demand their engineer lie, they did it in writing. The engineer is not the problem here.
> Lying is not an embellishment or puffery, it's a lie. Engaging a company for a 3 day pen test that's totally insufficient, that would be an embellishment.
I agree, but if the RFP question was phrased "have you done penetration testing?" then that leaves a lot of room for embellishment. If the question is "do you have SOC2 certification?" and you answer "yes" untruthfully, then that is a lie. If they ask for the SOC2 or pentest report and you give them a falsified document, that's where you're (probably) committing fraud.
> One of the most important part of pen tests is that they are external.
AWS/Google/etc have internal security teams doing their pen tests, so no, this isn't true.
> Just doing your job as an engineer and looking for bugs is not a pen test.
What about an engineer spending an afternoon running ZAP[0]?
> It's like saying, "what is an audit really? We have accountants and they check our books for anomalies."
Yeah, which is why you don't just ask a company "do you keep track of your finances?" if you're investing in them, you request external auditors.
I have literally worked alongside external pentesters for some of those organizations you allude to. I still remember their codenames.
They might have the scale to have internal pentesters nearly as isolated as external pentesters. A ten person startup definitely does not.
Regardless of what tools you use, an internal pen test isn't the same. Do internal accountants use different tools than KPMG? Probably not.
The RFP likely did contain more precise language.
I encourage you to reflect on your position. It's very odd to me that your attitude is, "if the customer didn't want to be lied to, they should've tied me down better, because I'm like a djinni who will twist your words against you if there's the slightest ambiguity." I understand the sentiment that RFPs and contracts need to be locked down. I don't understand the sentiment of, "they had it coming."
> Regardless of what tools you use, an internal pen test isn't the same
It isn’t the same as an “official” pen test for a 10 person company with non-specialists, sure. But the document, to our knowledge, didn’t ask if they had some specific form of pen test.
> because I'm like a djinni who will twist your words against you if there's the slightest ambiguity.
They aren’t twisting anything. If a quick and informal pen test meets their definition then they should be more specific.
> Just doing your job as an engineer and looking for bugs is not a pen test
A pen test’s goal is to find security bugs by posing as an attacker. There is no requirement that it is systemic, formally documented, performed by a “security expert,” or that it is done by any external party.
Those are all desirable _properties_ of a pen test that may be required for various certifications, but an engineer can absolutely conduct a quick and informal pen test at any time.
>>This situation reads like an engineer who needs a bit of business/sales experience more than anything.
Your post reads like a salesperson hand-waving away actual situations of consequences in favour of a quick buck. There is literally no embellishment in this scenario, it's outright lying. He didn't say "well we tested security in some things but it wasn't up to a great stabdard", he said "we didn't do any pen testing" and when asked he said the opposite.
lol there's no ambiguity here. I love it when otherwise "street-wise" salespeople get challenged with very obvious scenarios they all of a sudden become postmodern philosophers.
"I mean, what does it even mean to COMMIT fraud? I mean, did I really "commit" to it if I did it once but gave it up after? Hmmm? Ever ask yourself these deep kinds of questions?"
Give me a break. Some sales people are so deep into a near-sociopathic lifestyle of "sales" that they are just pathological liars in the most literal sense. They don't even see themselves weaving deception.
> he said "we didn't do any pen testing" and when asked he said the opposite
I conjecture they _did_ do something that could reasonably be called pen testing and didn’t realize it.
GP even gives some examples: testing authentication code, checking security group configurations, and testing API calls all counts as rudimentary pen testing.
My point is that while they don’t believe they did a pen test, it’s very possible that their standard correctness testing of security related features was sufficient to meet a broad definition of the term.
I don't actually like this answer but there's likely some truth to it.
I am reminded that when I bought a house in my twenties, at the scheduled closing there was some detail that was incorrect. There was a line in the document where we had to say something like "Yes, x paper is in hand." In reality, I think we were still waiting for it.
And when I was like "But this isn't true. Shouldn't we wait until we have this?" the banker said "That's your call."
So it was lie about it because the paperwork wasn't going to go through if it said anything other than "Why, yes, we absolutely have that!" or delay closing on the house, which could mean losing it. So I signed.
And it never came up again. No one ever called and said "But what about blah?"
If you are knowledgeable about the bureaucratic process, you may know which check boxes must be checked off, true or not, to file it and in most cases don't actually matter. If you aren't knowledgeable, you are seriously gambling.
So in reality, this kind of thing does go on, like it or not. And if you are too pedantic, you can't get things done. Things will grind to a halt while you dot every I and cross every T.
There’s a difference between you knowingly waiving a requirement that is to your benefit (receiving the necessary disclosure document before closing) to further a goal to your benefit and you making a statement on a form to the detriment of someone else.
In your analogy it would be as if the seller ticket the box saying that they’ve disclosed all deficiencies to you despite not having done so, to not jeopardize your willingness to close.
I don't recall what the document was but in real estate, a defect can also negatively impact neighbors or future owners. If the defect was the responsibility of the previous owner and not having the disclosure at the time of the closing means you no longer have legal standing to pursue remedy and you don't get around to fixing it, it may get grandfathered in and now it's potentially a permanent situation.
I think TFA is a case where the OP messed up. He shouldn't have lied to appease his boss and it seems he wasn't clear about that at the time.
He didn't seem to feel he was lying. He told his boss X then did as he was told. And I think this sort of mental disconnect is common and, among other things, leads to problematic choices for how to relate online.
In his mind, while filling out the papers, what he said was part of this larger conversation and his boss knew what he really believed to be true. The problem is that putting it in writing means other people can read it and they aren't part of this larger conversation.
I was clear that signing the papers could come back to bite me but based on the banker's remark inferred he wasn't concerned and he felt rescheduling was a bigger problem. Odds are good the item showed up soon after and was added to the file and said the "right" thing, so didn't create problems.
I still think what I said elsewhere: OP should probably be planning his exit. But I do also understand why some people have sympathy for his boss and feel like playing devil's advocate here.
If a client tells you what they need, you don't get to decide what the client needs.
The sort of "business/sales" you're referring to may work well for someone not interested in building long-term relationships for repeat customers, but you're describing the exact type of fratboy used car salesman that I avoid doing business with whenever possible.
If the engineer has a problem now, they'll have a complete mental breakdown during a standard SOC/ISO certification.
However, the director also does not seem politic enough to maintain plausible deniability, if they're saying "what the fuck, just sign your name uncritically on this statement" vs guiding them towards a long, detailed, qualified, technically-truthful answer. ("We did these things on these dates" and leave it up to client to evaluate whether its sufficient/is really pen testing) Which the client would require anyway if the process moves forward. The entire situation just seems like a shitshow.
I get what your saying, but boy do I not want to live in society created by this worldview. In my opinion, this approach is antisocial. It forces everyone to write massive contracts and build bureaucratic mega processes to validate and specify every single definition in that contract because you can't ever trust your counterparty to abide by the spirit of your request.
This comment really nicely captures how I feel about this. There's something to be said about good faith and knowing what the spirit of the agreement is.
There are some comments here saying stuff like "these compliance forms are ridiculous and are often just bureaucratic nonsense" and you see comments advocating for playing dumb and answering in bad faith and there you go.
I see there being a bit of an attitude of "everyone is doing it" to justify also doing it just to compete because you're at a disadvantage if you don't. And that's not entirely wrong but it sucks and I personally will avoid competing in that way. Probably that means not much sales in my career. Or science, but that's another topic...
I worry that this is how we all end up doing meaningless administrative work, while we outsource all the useful "actual" work to poorer nations, where all of our administrative specifications can be overlooked or ignored.
To each their own, I don't want to be the victim of identify theft because sales pushed engineering into releasing shit code with a simple buffer overflow that noone ever tested for.
Why does my worry about bureaucracy have to mean I want to be the victim of identity theft? I understand and agree with the value of good security practices. I just worry that assuming all software is insecure unless some very complicated and "iron clad" contracts exists and are independently validated. It's a recipe for a very inefficient society.
I actually worry that this mindset of adversarial relationships make it MORE likely for your identity to be stolen.
I don't think there is too much software a government agency will buy that I wouldn't like to see pen tested. By it's nature government is often dealing with sensitive data.
I'm not sure why having standards that should be met around software testing would make my data less secure. Weve seen leak after leak and so frequently it's some basic issue caused by massive incompetence, or more often, by decision makers cutting corners to make a quick profit.
Absolutely nothing you said in this post is true or valuable, other than your final sentence. You appear to be far too comfortable with lying to clients.
Remember that the goal of most security/procurement questionnaires isn’t _security_ … it’s _conpliance_
Compliance is not security.
For buyers, they often just need to check a box so that they can maintain their own compliance obligations. They probably don’t care all that much about your security if you can check all the boxes.
Pen tests are a joke. You can get perfectly acceptable pen tests (from compliance perspective) done for $2k in less than a week and the testers will try to find a couple things but their goal isn’t to evaluate your app’s security. Their goal is to make you happy with the report you’re paying for (and you won’t be happy if it has a list of 100 security bugs)
Note: I don’t like that this is how compliance works. I wish compliance were more closely tied to security. But that’s simply not reality.
Sure it's about compliance. In the event of a problem followed by a lawsuit, how comfortable would you feel explaining to a judge that you falsified the compliance forms? If the answer isn't 'completely comfortable", you probably shouldn't go around falsifying compliance forms.
If all they said was effectively “we did some pen testing at launch” without any claim of meeting any standard or going into much detail then even the most rudimentary test/validation of the correctness of any vaguely security related code could be charitably considered to be a pen test.
In some situations, maybe. In this case, given the director swore at OP, I strongly suspect foul play.
Were we talking some light fudging, the director would have spent time explaining what the customer is asking for and worked with them to determine whether what they do supports a statement of “we do security testing.”
And if they can’t support the claim? Well, the director’s response will tell you if they’re an honest broker or not.
Honestly none of that is a pen test under almost anybodys understanding of the word.
If the request wasn't specific, you may be able to get away with arguing that way in a court, who knows.
When you tell the director it is not true, author a document that says otherwise, and then document the whole thing on the internet, by that point it is a lie not embilishment, and I'd be worried about fraud myself.
The entity conducting the pen test -- i.e. a third party with no interest in shipping the product -- is what makes it a pen test. Otherwise it's just QA.
First, there's always a good chance nobody on the client side actually cares about this. RFPs tend to come with hundreds of questions, most of which are put in as requirements by completely detached departments to make them feel important, but don't actually matter. If something is important, it can be negotiated. For example, if the pen test is important, you can get the client to agree to sign the contract with a clause that says if they don't get pen tests results by X date the contract is void.
Second, well, what is a pen test, really? If you ever called one of your APIs to validate that your authorization or authentication code works, or you've validated that your AWS security groups are blocking external traffic to your database, congrats, you've performed a "pen test"! The client won't consider that sufficient, of course, but it's a justification for an answer on an RFP, at least. If they want more details, as they do in this case, they can always schedule a follow-up call or ask to see the pen test results document.
This situation reads like an engineer who needs a bit of business/sales experience more than anything. In their discussion with the client, you can start with "we've currently only done in-house penetration testing, but are looking to contract an external vendor to do penetration testing by X date." Assuming you've done any testing, this is a true statement, and the client can determine if that's sufficient.
The real solution to this is to get SOC2/ISO27001 certification, though, which makes a lot of these RFP headaches go away.