Isn't sending both the blurred and non-blurred picture over the network the way we did it since two decades in web dev? With (many!) high resolution pictures this is definetly less performant then a local computation, given that real networks have finite bandwiths, in particular mobile clients on spots with bad wireless coverage. It is astonishing what can be done with CSS/WebGL only these days. We needed a lot of hacks and workarounds in the past for that.
I don't have much data myself but when I was doing scraping some time ago I had thousands of examples where f.ex. the full-res image was something like 1.7MB and the blurred image was in the range of 70KB - 200KB, so more or less 7% - 11% of the original. And I might be lying here (it's been a while) but I believe at least 80% of the blurred images were 80KB or less.
Technically yes you could make some savings but since images were transferred over an HTTP-1.1 Keep-Alive connection, I don't feel it was such a waste.
Would love to get more data if you have it, it's just that from the limited work I did in the area it did not feel very worth of only downloading the high-res image and do the blur yourself... especially in scenarios when you just need the blurred image + dimensions first, in order to prevent the constant annoying visual reflow as images are downloaded -- something _many_ websites suffer from even today.
Given how nice system programming languages we have these days, I refrain to let classic Null-terminated C-Strings entering my program. Even on embedded programming we opt-in for std::string (over Arduino's String). I am just happy to save our time in favour of having some X percentage less optimal code.
It is seriously unfortunate that C++ managed to standardize std::string, a not-very-good owning container type, but not (until much, much later) std::string_view, the non-owning slice type you need far more often.
Even if Rust had chosen to make &str literally just mean &[u8] rather than promising it is UTF-8 text, the fact &str existed in Rust 1.0 was a huge win. Every API that doesn't care who owns the textual data can ask you for this non-owning slice type, where in C++ it had to either insist on caring about ownership (std::string) or resort to 1970s hacks and take char * with zero terminated strings.
And then in modern C++ std::string cares about and preserves the stupid C-style zero termination anyway, so you're paying for it even if you never use it.
> And then in modern C++ std::string cares about and preserves the stupid C-style zero termination anyway, so you're paying for it even if you never use it.
I don't think this in itself is a real problem. You pay for the zero at the end, which is not much. The real cost of zero termination is having to scan the whole string to find out the size, which with std::string is only needed when using it with C-style APIs.
If you want to pass the string_view to some API that expects NULL terminated strings, then a copy is necessary (well, maybe is some cases you can cheat by writing a NULL in the string and remembering the original character, and then after the API call restore the character).
This isn't as much a fault of a string_view type of mechanism, but rather API's wanting NULL terminated strings. Which are kind of hard to avoid on mainstream systems today, even at the syscall interface. Oh well..
Sure, but the thread here was about the forced NUL-terminator in std::string and the costs associated with that. If you want a NUL-terminator (e.g. for use with a C API) then you have to pay the copy (and in the general case, allocation) cost for substrings no matter how your internal strings look (unless you can munge the original string) and std::string is exactly the right abstraction for you.
But yeah, it would be nice if the kernel and POSIX APIs had better support for pointer+size strings.
> And then in modern C++ std::string cares about and preserves the stupid C-style zero termination anyway, so you're paying for it even if you never use it.
Is this required now? I've seen a system where this was only null terminated after calling .c_str()
c_str has to be constant complexity, so I guess the memory needs to be already allocated for that null character. I'd be surprised to see an implementation that doesn't just ensure that \0 is there all the time.
Ah, the system I ran into would've been pre-c++11.
Only saw it trying to debug a heap issue and I was surprised because I thought surely it's a null terminated string already right? They also checked the heap allocation size, so it would only reallocate if the length of string data % 8 was zero.
Facebook / Meta had their own string type which did this, turns out now you have an exciting bug because you end up assuming uninitialized values have properties but they don't, reading an uninitialized value is Undefined Behaviour and so your stdlib conspires with the OS to screw you in some corner cases you'd never even thought about because that saved them a few CPU cycles.
The bug will be crazy rare, but of course there are a lot of Facebook users, so if one transaction out of a billion goes haywire, and you have 100M users doing 100 transactions on average, the bug happens ten times. Good luck.
It seems C is going around in circles while everybody else has moved on
No, speed and "efficiency" are not a be-all, for-all.
Safety is more important than that except in very specific cases. And even in these cases there are better ways and libraries to deal with this issue without requiring the use of "official" newer C functions that seem to still be half broken
There's so much fiction regarding memory issues and limited memory issues and what to do if we hit limited memory issues when in practice terminate and give up is the (often enforced by the OS) norm.
I look forward to your solution to bridge these libraries to every other person’s slightly different implementation of the same library, which also has to talk to every other interface that cropped up over the last 50 years and takes null-terminated strings anyways.
Have fun playing with your Fischer Price KidSafe™ playset? >:)
At some point we are just outlining in general terms what we want from a language. If C was a toolbox it is a limited number of essential tools, other languages add so many things Alton Brown would faint from the unitasking nature of them.
I could do pretty much whatever I wanted in (DOS) user space with Pascal.
I can do pretty much whatever I want in a modern OS user space with whatever lang I prefer. "Oh but you might need C bindings" because the OS was build that way! (And with Windows/COM you might prefer a C++ bindings - just saying ;) )
> If C was a toolbox it is a limited number of essential tools
C is an old toolbox where 1/3 of them is a rusty finger-remover, 1/3 is a clunky barely do nothing metal crap and 1/3 of them kinda works
I'm all for a simplified set of essential tools, but not one where it's sharper on the user handle than it is on the business end
Null terminated C strings are still terrible for FFI. Pointer and length is a better solution and it is trivially interoperable with code that uses, say, string_view.
I haven't seen end-of-line conversion problems (as well as Unicode BOMs) for decades. My guess is that software quality improved these days. MS Notepad was a noteable tool which always mocked around with unix-style line endings. Using dos2unix and unix2dos utilities was something I commonly used in the early 2000s, in particular on dual boot computers. This was also the time where UTF-8 was not yet so widespread, but that is another topic ;-)
I can easily believe that it's gotten better, but I've hit it within the last 6 months; Microsoft Azure DevOps git[0] has a web editor that defaults to Windows-style EOL, and if you use it to deploy files to boxes running a Linux distro then some tools will completely break on config files that aren't using unix-style EOL. Ask me how I know. That was fun to fix. For bonus points, some programs on Linux are compatible with either line ending-_-
[0] I expect this isn't its real name, but it's MS so that's a lost cause.
Sad to say, I filed a bug in an internal tool just two weeks ago relating to how it processes Windows linebreaks. The tool is written using modern languages (Python 3.11) and frameworks (FastAPI).
It's hard to believe for me how you would not start buying own hardware at this scale. In particular when the hyperscalers (at first glance) don't have anything to provide to match the needs.
The biggest x86 machine I found tops out at 960 cores, but I'm not sure what exactly they need, if having more cores would solve their problems or would only make some other pipe burst.
To figure that out, we'd need to look deep into what's happening in the machine, down to counting cache misses, memory bandwidth usage (per channel), QPI link usage (because NUMA), and, maybe, even go down to the usage stats of the CPU execution units.
When they mention a lot of what was stored procedures has been moved to external web services, I get concerned they replaced memory and CPU occupancy with it waiting for network IO.
I would hazard a guess that they're not really CPU bound.
Assuming the poster Aussiepete80 is Australian I should point out that the much higher salaries in the US and the favorable E3 visa has largely brain drained Australia of their best and brightest. This guys army (dozens) of DBAs is likely the residual.
Not really a nice thing to say - not everyone is inclined to move to the US (for me, and I'm not that cool, it'd need to be more than 6 digits). I suspect there is plenty competent people wishing to stay in Australia (in fact, a lot of healthcare folk here in Ireland have decided to move there - weather is so much better, despite the giant spider and murderous fauna problem).
Predators here in Ireland are terrifying. The other day I was almost adopted into a family of feral cats.
One competent person probably, 'some' possibly, 'plenty' defiantly not. The general advice for tech people in Australia is to move to the US ASAP - the more competent the tech person the stronger the advice. The insidious effect of brain drain is the effect is self reinforcing, i.e. the more it happens the stronger the pull. In tech it's important to work with other competent people. Historically Australians are unusually good programmers, so I am not saying Australians are unskilled, but I am saying their best and brightest leave Australia ASAP.
Plus I would submit the reddit thread 'Biggest CPU for the Bad System' as evidence to my point. How could this person possibly think that 100s of CPU cores are the bottleneck and the solution is more CPU cores - how could the dozens of DBAs not figure it out - the amount of work that can be done in 100s of CPU cores is utterly insane and the fact that they didn't rule out obvious IOWait bottlenecks in their initial posts suggest that none of them know what they're doing. I guess they figured that Azure would take care of all of that.
The initial step would be to bring that computer in house and design that computer around the problem - Azure is not going to be open and honest about their bottlenecks. Last I checked Azure was big on their Network Attached Storage and Managed Disks which add quite a lot of latency and throughput bottlenecks compared to a PCIe 5.0 Enterprise SSD.
I’d say the telltale sign is picking SQL Server in the first place.
If you think there is a risk your database will grow to be enormous, it’s likely it’ll be the worst possible choice, with the possible exception of Oracle and DB2 on AIX (but, on the last two, at least you have fewer limits to scaling up).
Could also be a B-player hires C-players situation, with borderline incompetence trickling down from the C-level.
Certainly, if you're planning on outgrowing a SQL Server you should probably DIY tailor made indexes and query languages/engines. LLVM makes DIY languages much easier to make than they used to be and very with fast interpreters and AOT it's very performant.
The reason for my brain drain statement is that people don't understand how pernicious the brain drain is in Australia. The Australian government believes that there isn't a net brain drain because more people with degrees immigrate than those emigrating. But people are not substitutable cogs and the very smart are very rare and it is the loss of those few very smart Australians to the US which is so devastating.
If they need really 1600 cores, perhaps they should be looking at a cluster with an Infiniband connection? Infiniband isn’t cheap but they have to beat nearly a million dollars per month which gives them a lot of headroom.
They are also tied to MS SQL Server, it seems. Not sure it supports the fanciest gear out there. I've only seen Infiniband in HPC settings which are overwhelmingly non-Windows.
Another comment pointed out that the Reddit poster was probably just making the whole thing up, the account had made a bunch of different posts indicating they had a bunch of incompatible high-ranking jobs. The creative writing on that site is very odd.
I frequently read over virtually all of the docs of a language to grasp the ideas the authors had. Understanding a language theoretically is more of interest for me then tinkering around with it.
I love LibreOffice and use it everyday. However, IMHO, the LibreOffice homepage is not very advantageous. I thin it should advertise the product features and UI instead of the organization. Replace the damned stock photos with screenshots :-)
I do agree though that their webpage does not make it plain and clear what they do or who they are.
It's clearly made for people who already know what they are there for and not to draw new people into the fold, but to be fair, LibreOffice is not designed to be pretty and flashy, it's to get work done and to work with just about anything you can throw at it, so it makes sense that the webpage is the same.
Graphviz is a classical open source code and a core building block of many contemporary data science codes. There exist bindings in virtually any programming language and the software is by no means outdated, despite there are more fancy, maybe more interactive tools for graphing. Or niche products for visualizing extremely large graphs, for instance.
Just to give an example, GraphViz does a great job at SVG rendering in an Ipython/Jupyter browser based notebook.
reply