

Ask HN: A Microsoft engineer is hoping to hear your devops pains - brendanp

Hey Hacker News,<p>I&#x27;m an engineer at Microsoft in the Cloud and Enterprise division - one of the happy outcomes of the last few years is that we now have the freedom to talk directly with people about the problems they&#x27;re facing.<p>An area in which we think we&#x27;ve potentially underserved customers is in the devops arena - many of our toolchains are tuned for The Old Ways, and sometimes don&#x27;t offer the flexibility&#x2F;composability of alternate solutions.<p>It&#x27;s typically pretty easy for us to talk with people who already use our stuff; we have conferences, hosted forums, etc, that give us insight into how to tune things we&#x27;ve already built for existing customers.  What I&#x27;m hoping for are some thoughts from people who don&#x27;t use our tools, even if it&#x27;s for philosophical rather than practical reasons.  You certainly don&#x27;t need to be a Windows user - we&#x27;re not wed to any particular technology; we just want to build things that people find useful.<p>So, what prevents you from practicing continuous delivery in the fashion that you&#x27;d prefer? Are there areas that require constant investment to keep them functional? Do security or compliance concerns slow down your pipeline?  Are there specific investments that you&#x27;d want us to make to help ease your pain?<p>If you&#x27;re up for talking to us directly, we have a surveymonkey survey [https:&#x2F;&#x2F;www.surveymonkey.com&#x2F;s&#x2F;ND3ZRY9] that asks a few questions - we&#x27;ll get back to you and setup a phone call.<p>Thanks!
======
spdustin
First, my bona fides: I own a long-standing SharePoint training and consulting
company, was one of the first SharePoint MVPs (there were only two of us in
the first award cycle for STS), and I continue to devote my professional life
to teaching about and extending SharePoint.

What I like: PowerShell. Nearly everything about it. Composability and
grammar, pipelining, introspection ... It's a wonderful thing. Installing
SharePoint via PowerShell is a system automator's dream, IMHO.

When you're using the Windows machine it's on. Or using objects with remoting
support baked in. Or have configured PowerShell remoting, which is a bit of a
security black box for me to understand, and also, still requires Windows.

Now, the devops dream, one I think is shared by more people than are willing
to speak up about it. I know there are third party apps that enable this
dream, but they're so unknown and themselves have security issues, so devops-
focused folks haven't embraced it. But with this dream, envisioned with the
below sample fictional exchange from my terminal, a new world is open. One
with Ansible/Chef/Salt/Vagrant/etc singing in the choir. One with GitHub-
hosted repos proclaiming how a new Capistrano plugin will deploy their new
ASP.Next app from their Mac development environment to their Windows IIS host.
One where I and others can say, "hallelujah, hot damn, now we're talkin'!"

    
    
        $ ssh user@windowsserver.example.com
        Connecting to windowsserver.example.com...Connected
        User:PS>

~~~
Terretta
> _shared by more people than are willing to speak up about it_

I'll speak up about it. WinRM is cake, but the cake is a lie.

DSC has potential. Nano and DSC together with spdustin's request is nirvana.

@brendanp, you should have shared contact info in your profile. Since you
didn't, check my profile, check my name in Microsoft press releases[1] for
bona fides, then hit me up because we are working on this aggressively and
have just recently been talking about reaching out to you on this topic.

And to anyone else who thinks this is a cool space, email me, I'm hiring.

1\. Read between the lines of my second quote here:
[http://news.microsoft.com/2000/06/12/microsofts-new-
digital-...](http://news.microsoft.com/2000/06/12/microsofts-new-digital-
broadcast-manager-enables-next-wave-of-e-commerce/)

~~~
brendanp
Sure - I'm (perhaps predictably) brendanp at microsoft. I'd be happy to talk
more.

thanks!

------
stephengillie
I'm hardly your target audience, but I just left a .NET shop. Devs had trouble
using TFS and all of the new tools (like New Relic) because the majority of
their web app was written in VB. Due to bad design decisions, like circular
DLLs, the application won't even compile properly in TFS. They're in the
middle of replacing it with a ground-up .NET 4 rewrite.

And architecture choices, like creating a home-grown module for URL rewrites -
and then later, when the URL Rewrite module was released with IIS8, choosing
to continue using the home-grown module instead of the MS official module.

The same shop had a data import application, to ingest and ETL data from
hundreds of sources. Likewise, they were heavily hamstrung in that most of the
ETL was written in the depreciated DTS format, not something any modern SSIS
can work with.

So...I guess my only answer is one you probably aren't really interested in -
what's held back devs in my space has been old, depreciated code that nobody
makes tools for anymore.

~~~
brendanp
I'm interested in everything!

In this case, what delayed updating the web app to a more modern architecture?
Company culture? Lack of resources?

It does seem like pipelines need not just to be maintained, but also
continuously improved, in order to avoid being left behind by the rest of the
tool ecosystem.

~~~
stephengillie
Company culture was a huge part of it - lack of owner interest, previous
architect apathy, intense micromanagement, ticket/maintenance focus encouraged
small targeted fixes instead of necessary rewrites, circular DLLs heavily
complicated development.

And brain drain - not many developers (or engineers or anyone else) willingly
stay in environments like this. So the people who have spent years learning
the application and how to maintain it find other work, and are replaced by
people who have to analyze the system anew.

\---

A little more targeted:

\- TFS 2013 has AD integration, but it works in the dumbest way possible - you
can't just add someone to an AD security group and they get TFS permissions,
nope you have to go into TFS and find their AD account and add them.

\- It's difficult to debug websites on IE8 because it doesn't have modern
debugging tools. Would it be possible at all to have a browser release with
the IE8 engine and the IE11 debugging tools? (I'm currently working at a job
with an IE8 dependency on a web app, so I understand why it's still around.)

\- Sharepoint Online is a decent document repository, and TFS is a decent
document repository - both have their advantages and disadvantages. But they
in no way integrate. This caused major issues when I was providing Ops
documentation from a Sharepoint Online site, trying to work with Devs putting
their documentation in our local TFS. (Working with either in Jira is beyond
painful)

\---

Unrelated, I wish WDS and WSUS were more integrated - when I push an image
from WDS, I want it to have all security patches slipstreamed in from WSUS.
Instead, I deploy an image with one and use the other to patch it.

------
toomuchtodo
Can you edit your post? You may want to make specific mention that you're
talking about the MS toolchain. The open source toolchain for CI and
integration with Linux/FreeBSD hosting/cloud providers is sufficiently mature
to not run into the pain points you describe (build systems, containerization
through lxc and docker, aws and every other cloud provider's api, and so
forth).

Disclaimer: Infrastructure engineer who does DevOps as well.

~~~
brendanp
I'm definitely not trying to limit things to the Microsoft toolchain. I
recognize that composability is super important, and anything we build should
participate in the greater ecosystem.

------
Nelkins
One feature I'd really like to see would be the option of deploying a single
binary (even if it's not a native binary; that would be really great, and I
know the LILC effort is working toward something like that[1]). I know you can
sort of accomplish this[2][3], but it would make me really happy if there were
first class support for it.

[1]: [https://github.com/dotnet/llilc/](https://github.com/dotnet/llilc/) [2]:
[http://research.microsoft.com/en-
us/people/mbarnett/ILMerge....](http://research.microsoft.com/en-
us/people/mbarnett/ILMerge.aspx) [3]:
[http://blogs.msdn.com/b/microsoft_press/archive/2010/02/03/j...](http://blogs.msdn.com/b/microsoft_press/archive/2010/02/03/jeffrey-
richter-excerpt-2-from-clr-via-c-third-edition.aspx)

------
leap_ahead
Drop the prices for Azure VMs to the reasonable level similar to that of the
other hosting companies. Right now the first tolerable configuration A1 costs
$57 and this can be had for $15 elsewhere.

Please also make your payment processor accept virtual credit cards so I can
actually pay you money with the means I have (not in a possession of a real
credit card). PayPal will also do nicely.

~~~
brendanp
I don't have much insight into Azure pricing, unfortunately. You need the
bigger disk with the standard tier A1 for your app/service?

~~~
leap_ahead
No, the disk is fine. I just wish that VM cost $20, perhaps $25, but not $57
which it costs now (I've corrected my original post, it was not $67 but $57 as
I remembered wrongly).

I wish to publish a relatively simple app for a reasonable price and I'm sadly
skipping Azure from my consideration. I'm seeing VMs similar to A1 cost about
$12-15 with many hosting companies. And for $20-25 I can get your level A2
which costs $115.

I realize you may not be in a position to influence pricing. I just would like
to point out that the pricing of Azure VMs is not simply uncompetitive, but
plainly prohibitive. I hope Microsoft does something about it.

Saying it as a veteran Microsoft developer since the 90s.

~~~
janpieterz
I think the lowest entry point for this is an A0, which offers an admittedly
very small (half an A1) VM for 9.98 euro a month. Good to start off with, and
if you build your application slightly differently you can actually run it on
two machines, total power is the same, with only the communication overhead
between them which luckily isn't that high if you put them in the same region.
I find that I can run a very decent amount of software on Azure for a very
very low price, but sometimes I needed to make certain architectural changes
(for example using Cloud Services + Table Storage + Service Bus instead of
VM's and SQL Server makes the whole system very cheap).

A very nice thing that NServiceBus does on Azure is giving the opportunity to
have multiple endpoints hosted in the same Cloud Service [1]. Taking this
mindset, for small applications, you can build it so when needed (and
presumably when money is also less of a thing since you need more), you can
scale out easily, but when just starting up you can do it super cheap. This
depends a lot on your application though, but for example the Topshelf
framework [2] can help a lot.

I'm not affiliated with either Microsoft or NServiceBus, but I am an avid user
of both and really love the ecosystem, including the pricing ;)

[1] [http://docs.particular.net/nservicebus/azure/shared-
hosting-...](http://docs.particular.net/nservicebus/azure/shared-hosting-in-
azure-cloud-services)

[2]
[https://github.com/Topshelf/Topshelf](https://github.com/Topshelf/Topshelf)

------
Eridrus
The thing keeping us from CD is a lack of faith in our automated testing,
mostly because we don't have enough automated testing, but partly because our
problem domain makes it very hard to automate the testing that we are really
concerned about doing. Part of this is definitely lack of testing culture
though.

~~~
brendanp
Yeah - I'm sympathetic. It's often difficult to sell people on the notion that
integration testing might be just as expensive to implement as the
service/feature that you're building in the first place.

It sounds like this is partly an education issue - are there specific areas
within your problem domain that more tooling would actually help with?

~~~
Eridrus
Our fundamental problem is that we have a Typescript (<3) analytics payload
that has to deal with lots of weird JS execution environments that we
fundamentally can't recreate in a lab, which means we have no idea if
collection code is working as intended until we run it in prod and have a
manual look at the data.

But I started thinking about this in a bit more detail, and the things that
make our problem domain hard don't necessarily have to block doing fast
releases, since the failure scenarios that we want to block should largely be
automatically detectable.

The parts that we haven't been able to automate though are around verifying
that our payload, does not have any user visible impact on sites we run in,
including DOM elements we want hidden definitely still being hidden,
verification that we don't trip any SSL or other warnings, and we don't
generate console warnings from the browser, etc. So if Selenium was not a
complete hack job and had more knowledge of the browser chrome/UI, we could
probably fully automate it.

------
penguinlinux
Hello and Thanks for reaching out to the Dev and Operations community. I am a
devops engineer who is very familiar with Amazon AWS, Backspace Cloud and
Google Compute Engine. Is there a way that we can get free trials of Amazon
azure. AWS provides you with free credits for a year to run a few services and
try their APIs and also try their services. This has been useful because it
allowed me to learn about their products without having to pay for them. Then
I have been able to use my knowledge and promote AWS service at companies I
have worked and pay for AWS.

Is there such a resource with Microsoft Azure?

Thanks

~~~
janpieterz
Hi,

I'm not affiliated with Microsoft, but you can sign up for a free trial [1].
They'll give you 150 euro in credits for a month to play around with. Besides
that, a lot of their services are available very cheaply to play around with,
for example table storage and blob storage, Azure websites, would hardly cost
you anything or you can even use it for free if you hardly have any data/usage
to get to know the system. There are of course things that will cost you, but
if you're just willing to play around, for 50 euro a month investment yourself
you can get quite far.

What Microsoft also offers is Bizspark [2], which if it is applicable, will
help you along the way nicely.

I fully agree that the Amazon credits is setup a little bit nicer to just play
with, but with that first month (if you dedicate some time for it) you should
get a good feel of it, plus the opportunity for Bizspark helps a lot, and if
you don't fit in the Bizspark requirements I think it's more than reasonable
to assume you can pay 50 euros a month to test it for a longer period than a
month!

[1]
[http://www.microsoft.com/bizspark/default.aspx](http://www.microsoft.com/bizspark/default.aspx)

[2] [http://azure.microsoft.com/en-us/pricing/free-
trial/](http://azure.microsoft.com/en-us/pricing/free-trial/)

------
Someone1234
You should seriously re-post this on Reddit's /r/sysadmin there are tons of
DevOps people in there.

~~~
brendanp
That's a great idea, though I'm a little worried about being perceived as an
astroturfer.

~~~
Someone1234
You could try contacting the mods first, state your reasoning and ask for
their blessing?

------
antod
I work in a small heterogeneous Linux/Windows SaaS environment where most
coding is done in Java or Python. We don't use an MS toolchain because
historically MS tools want you to use other MS tools almost exclusively.

But in terms of DevOps things MS as a whole could do that would make our life
easier:

* Contribute to Vagrant, Packer, Saltstack and Ansible etc to help make Windows clients better first class citizens with those tools.

* Keep shrinking the disk and deployment (both time and space) footprints of Windows Server and SQL Server. And making sure they stay close to that original size after extensive patching without ballooning out.

* Somehow make licensing and activation etc less painful.

* Native SSH and rsync servers and clients shipped with Windows Servers for better interoperability. No need for a posix command shell, we'd be happy with executing Powershell over SSH - eg if remote Powershell had an SSH transport option as well as WinRM. And being able to forward ports over SSH too.

* Make high availability, robustness and security features common across SQL Server editions. We don't care about business intelligence, reporting etc etc but do want things like mirroring and compressed backups etc without paying through the nose for it.

------
neduma
My 2 cents. (By the i really love the recent MS efforts - When was the last
time heard M$? Can't think of it..)

I heard a lot of good things about Powershell. Why don't you start from there?
I see it is a 'carrot' hook. Build an friendly/developer centric ecosystem
based on that and build it from there..

If you really look at vagrant/docker/ansible ecosystem, It's all about
cli/api/scriptable and disposable environments. Can't we come up with some
kind of framework (called Ultron, for example) which nicely wraps around
vms/containers/cmt tools based on powershell semantics with full integration
of Azhure. I would love to check that out.

Any effort to bring Azure into developer desktop/laptop to play around would
be compelling for devs.

------
aprdm
I am another one that also enjoys the recent microsoft effort to open source
and be more open with the community. I think devops envolves a lot of
scripting, having a POSIX compatible shell out of the box in Windows would
help a lot. Same for a decent ssh client.

~~~
brendanp
Thanks for the suggestions! We've definitely heard this feedback consistently
- especially around SSH.

------
jurymatic
There are some things we've encountered that should have been relatively basic
but turned out to be a nightmare. One is configuring a VM to have a public,
static IP. I currently have a Windows VM running that somehow has no fewer
than 3 different static IP's, depending on where you look. Yes, we followed
the directions and had our dev ops guy give it a try.

Second, we deal with some third party vendors who require us to white list our
IP's. We ended up having to config a VPN through Digital Ocean because even
our dev ops guy and our IT guy together couldn't figure out how to route our
traffic.

------
lawnchair_larry
Test 1:

Install 3 systems with a base OS install, and nothing else - Windows, Mac,
Linux.

Put a programmer in front of each. Race to create and run "Hello World" in
Python, Ruby, C/C++, and Java.

Test 2:

Set up a Windows Server and a Linux server running a web erver. Using a Mac or
a Chromebook, remotely edit the web server configuration file over a dialup-
speed connection.

Test 3a:

Get a fresh installation of Windows, Linux, Mac. Race to get up and running on
a local instance of Django and Rails using a sqlite db.

Test 3b:

From the previous test, swap out sqlite for postgres. Then try swapping it out
for SQL server.

Test 4:

Try to do even 1 single thing in powershell against a remote Windows Server
when using a Mac or Linux.

Test 5:

Without syncing, search for a function by name across your repos hosted on
Github/Gitlab/Gitweb. Try the same thing on Visual Studio TFS or Visual Studio
Online.

Test 6:

Delete HTTP.SYS on a Windows Server and try to do anything at all. You can't
even use remote powershell! There is no way to get up and running with a
simple userland webserver using anything in your entire ecosystem. All routes
lead to WCF and HTTP.SYS. Can you imagine if you needed a linux kernel module
loaded in order to use SSH?

Test 7:

Install Windows, Linux, Mac on the first Wednesday of the month. Assuming
updates are released for your major services on all three platforms that month
- survive until the following Wednesday without a reboot.

Test 8:

Survive the Hello World test in #1, using C#, without a reboot.

Test 9:

Write a simple script to query a TFS server from Mac/Linux. Write a simple
script to query a redmine server from Mac/Linux.

Test 10:

From a fresh install, capture a single packet over loopback on all platforms.

Test 11:

Deploy a repo server for all of your internally developed packages, and have
any other developer fetch and install that using a native package manager and
a single command.

Test 12:

<I have nothing here, but it should be something to demonstrate to how
cumbersome it is to deal with manipulating XML for everything, compared to any
other text-based format>

Test 13:

Try to integrate Gitlab with whatever Oauth2 thing Azure provides. Compare
this with trying to use oauth2 via Google or anyone else.

\--

By doing these things, you'll find that they are nearly frictionless on the
other operating systems in comparison. You'll find that some are actually
impossible to do on Windows.

Actually, you'll have them all finished for Mac and Linux before Visual Studio
is finished installing in preparation for test #1.

------
ramon
Time to think about cloud, virtualization and containers when talking about
CI. The scenarios I see currently are: 1) AWS - S3 / EC2 / Cloudfront 2)
Containers (Docker) 3) Virtualization (Citrix, VMWare)

~~~
brendanp
Just to make sure I understand, you're suggesting that we need to ensure that
our tools fully integrate with non-Microsoft cloud providers, container
providers and virtualization providers (rather than being primarily targeted
at Azure/Windows containers/Hyper-V)?

~~~
munishm
I don't think that would be feasible for something like MSFT as they can loose
their market control but it would be a dream come true for devOPs - switching
between the Clouds or using S3/SQS with Azure.

