Hacker News new | comments | show | ask | jobs | submit login

how does convox scale when moving to non-trivial aws scenarios? if i have three autoscale groups running seven different apps with associated kinesis streams, s3 buckets, elasticache, rds and elasticsearch instances how cleanly does convox handle this? what about monitoring and reporting?

We are working to normalize the AWS scenario for all apps.

Every Convox cluster is an autoscale group managed cluster of ECS instances.

Every Convox app gets its own Kinesis stream for logs, ELB for load balancing, and S3 buckets for settings, build artifacts and encrypted environment. And the app processes are run via ECS.

So I'm confident we could handle the 7 apps in a single cluster, scale the cluster instance size and count, scale any individual app process type, and handle any individual app load balancing or log throughput.

You can provision some services like RDS with our tooling which make it really easy to link to apps. You can also bring your own services like elasticsearch or pre-existing RDS and set them in an app's environment to use it.

There are a couple monitoring tools built in.

We automatically monitor AWS events like ECS capacity problems and send Slack notifications.

You can also use our tooling to forward all your logs to Papertrail and configure your searching and alerting there.

More CloudWatch Logs and Metrics work is coming in the near future.

i appreciate you taking the time to reply, thanks.

some additional questions though:

1. are security groups opaque within convox or are they exposed to developers?

2. when you say monitoring tools are built in and you have tools for logging does this lock me in to the convox log pipeline and monitoring? what i want to use sensu on my instances? do i have to add sensu to every container? if you run, for instance, five containers per vm do i pay the overhead of five seperate sensu instances? same question for something like logstash?

3. you mention vpc. aws has proven stingy with vpc service limit requests in the past. i have trouble getting them to grant more than low double digits per region/account. can i run multiple convox racks per vpc or is the one vpc per rack a hard requirement?

1. Devs don't have to worry about security groups with `convox install && convox deploy && convox ssh`. But security groups are created with CloudFormation and you can use your AWS keys to introspect and change them.

2. Currently every app gets a Kinesis stream and we tail all Docker logs and put them into Kinesis. Then `convox logs` can stream logs from Kinesis, and `convox services add papertrail` adds a Lambda / Kinesis event source mapping to emit the stream as syslog to Papertrail.

I'm pretty happy with this setup and think it represents a good default infrastructure that is still extensible.

Would Kinesis -> Lambda -> Sensu make sense too? It's a pretty new pattern but this seems a lot saner to me than per-container log agents, or even bothering with custom logging drivers.

That said, one user has been using logstash by bringing a custom AMI with his logstash agent and creds baked in.

3. It's one VPC per rack, but I could see modifying that. We've already started to parameterize some VPC settings like the CIDR block to help integrating with your existing VPC usage.


Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact