Hacker Newsnew | past | comments | ask | show | jobs | submit | rishiloyola's commentslogin

No I am not using Kafka. It is just basic python server. I want to calculate what is the nth percentile of size of my incoming request object over past one hour.

I don't want to store size of each request in memory. It will eat so much of my RAM.

Incoming traffic:

- 1st batch

--> 60 requests

---> size of 1st request is 10kb

---> size of 2nd request is 2kb

...

...

- 2nd batch

--> 10 requests

---> size of 1st request is 5kb

---> size of 2nd request is 8kb

...

- 100th batch

I am talking about percentile(10th, 50th, 95th) size of request.


How do you handle back pressure?

Do you have specific module for it?


Well logstash is now supporting good persistent queue. https://www.elastic.co/guide/en/logstash/current/persistent-...

I don't know why author didn't put correctness tick mark on it.


It's very likely we're doing something wrong with this test, but after many hours of trying we couldn't get our simple test to pass for Logstash, even though it passed for others:

https://github.com/timberio/vector-test-harness/tree/master/...

Definitely open to feedback on what we're doing wrong.


This is totally off topic, but holy crap, first message in 11 years. I would have lost my password about a dozen times by now.


That answer was definitely worth the wait.

(there's a "Forgot Password" link somewhere here).

[do you always look at poster's activity? sounds time consuming.]


yeah not even their blogs.


So finally did you impress that girl or not? :D


No... but some people on the internet thought it was cool, so I can live with that :)


You impressed the Internet and that is a greater accomplishment than impressing a girl.


This is what I try to tell myself


You okay, friend?


Perhaps not?


Not able to access it. Nginx error - 502 Bad Gateway


No, you are not the only one here :-)

Yeah, sometimes I am really pissed off because many people just try to find issues in my PR. They don't care for the rest of the work. I would suggest you talk to your manager about this topic. You need a better environment and which is fine.


Yeah, the harsh truth.


Hey I am interested in working with you remotely part-time(20-25 hours). How to contact you? Mention your email or website.


Please read the question properly before writing your answers.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: