What seems missing:
I can not see the answer from the different models.
One have to rely on the "correctness" score.
Another minor thing: the scoring seems hardcoded to:
50% correctness, 30% cost, 20% latency - which is OK,
but in my case i care more about correctness and latency I don't care.
Wow! This was my testprompt:
You are an expert linguist and translator engine.
Task: Translate the input text from English into the languages listed below.
Output Format: Return ONLY a valid, raw JSON object.
Do not use Markdown formatting (no ```json code blocks).
Do not add any conversational text.
Keys: Use the specified ISO 639-1 codes as keys.
Target Languages and Codes:
- English: "en" (Keep original or refine slightly)
- Mandarin Chinese (Simplified): "zh"
- Hindi: "hi"
- Spanish: "es"
- French: "fr"
- Arabic: "ar"
- Bengali: "bn"
- Portuguese: "pt"
- Russian: "ru"
- German: "de"
- Urdu: "ur"
Input text to translate:
"A smiling boy holds a cup as three colorful lorikeets perch on his arms and shoulder in an outdoor aviary."
$ time target/release/fetch_and_render "https://www.lauf-goethe-lauf.de/"
real 0m0,685s
user 0m0,548s
sys 0m0,070s
$ time chromium --headless --disable-gpu --screenshot=out.png --window-size=1200,800 https://www.lauf-goethe-lauf.de/
real 0m1,099s
user 0m0,927s
sys 0m0,692s
# edit: with a hot-standby chrome and a running node instance a can reach 0,369s seconds here
Obligatorische Pastete:
"16GB Ram sind Flischt, ohne wenn und aber. ECC ist nicht Flischt aber ZFS ist dafür ausgelegt. Wenn in Strandnähe Daten gelesen werden und es kommt irgendwie was in den Arbeitsspeicher, könnte eine eigentlich intakte Datei auf der Festplatte mit einem Fehler "korrigiert" werden. Also ECC ja. Das Problem bei ECC ist nicht der ECC-Speicher an sich, der nur wenig mehr als konventioneller Speicher kostet, es sind die Mutterbretter, die ECC unterstützen. Aufpassen bei AMD: Oft steht dabei, dass ECC unterstützt wird. Gemeint ist aber, dass ECC-Speicher läuft, aber die ECC-Funktion nicht genutzt wird. LOL. Die meisten MBs mit ECC sind Serverboards. Wer nichts gegen gebrauchte Hardware hat, kann z.B. mit einem alten Sockel 1155-Xeon mit Asus-Brett ein Schnäppchen machen. Ansonsten ist die Asrock Rack-Reihe zu empfehlen. Teuer, aber stromsparend. Generell Nachteil bei Serverboards: Die Bootzeit dauert eine Ewigkeit. Von Consumerboards wird man mit kurzen Bootzeiten verwöhnt, Server brauchen da oft mal 2 Minuten, bis der eigentliche Bootvorgang beginnt. Bernds Server besteht also aus einem alten Xeon, einem Asus Brett, 16GB 1333Mhz ECC-Ram und 6x 2TB-Platten in einem RaidZ2 (Raid6).6TB sind Netto nutzbar. Ich mag alte Hardware irgendwie. Ich reize Hardware gerne bis zum Gehtnichtmehr aus. Die Platten sind auch schon 5 Jahre alt, machen aber keine Mucken. Geschwindigkeit ist super, 80-100MB/s über Samba und FTP. Ich lasse den Server übrigens nicht laufen, sondern schalte ihn aus, wenn ich ihn nicht brauche. Was noch? Komression ist super. Obwohl ich haupsächlich nicht weiter komprimierbare Daten speichere (Musik, Videos), hat mir die interne Kompression 1% Speicherplatz beschert. Bei 4TB sind das ca. 40GB Platz gespart. Der Xeon langweilt sich trotzdem ein bisschen. Testweise habe ich gzip-9-Komprimierung getestet, da kam er dann schon ins Schwitzen."
This has been discussed on HN some times before. User xornot looked at the zfs source code and debunked "faulty ram corrupts more and more on scrub", for more details see
https://news.ycombinator.com/item?id=14207520
I've sorted the colors by luminance/lightness and added a gray swatch for comparison so can explore which color pairs pass WCAG contrast checks.
I haven't really gotten into colorblind safe colors like this yet where the colors mostly differ by hue and not luminance. Colorblind and non-colorblind people should be able to tell colors apart based on luminance difference i.e. luminance contrast. Hue perception is impacted by the several different kinds of color blindness so it's much trickier to find a set of colors that everyone can tell apart. This relates to the WCAG recommendation you don't rely on hue (contrast) to convey essential information (https://www.w3.org/WAI/WCAG21/Understanding/use-of-color.htm...).
The gray swatch above could be called colorblind safe for example because as long as you pick color pairs with enough luminance contrast between them, colorblind and non-colorblind people should be able to tell them apart. You could even vary the hue and saturation of each shade to make it really colorful, as a long as you don't change the luminance values the WCAG contrast between pairings should still pass.
Sadly, it isn't strictly true. maybe the article should have said "applied" not "created".
Even the wikipedia page for "Usability" points to a 1982 BYTE article advocating "Usable" for software tools.
The existing word "usability" was being applied in human computer interface texts/papers around the time (1987, 1988) as computer UIs made advances.
The best summary of history I could find was in this article (section 5.1), claiming the first usage was in 1971, or 1979, or perhaps 1981, depending on interpretation.
If you want a single example it is untrue, a good example is "IBM makes usability as important as functionality" from 1986. I found a copy online but hate deeplinking to such things.
It's possible that Cachet was unaware of all that academia and independently "created" the term.
Of course the actual word is much, much older in the world of meatspace.
I was using xcopy at the time as a kid and still play with a physical Amiga. Nostalgia.
If you run the `validate.py` script available in the repo, you should see correlation numbers similar to what I've pre-tested & made available in the README: fssimu2 achieves 99.97% linear correlation with the reference implementation's scores.
fssimu2 is still missing some functionality (like ICC profile reading) but the goal was to produce a production-oriented implementation that is just as useful while being much faster (example: lower memory footprint and speed improvements make fssimu2 a lot more useful in a target quality loop). For research-oriented use cases where the exact SSIMULACRA2 score is desirable, the reference implementation is a better choice. It is worth evaluating whether or not this is your use case; an implementation that is 99.97% accurate is likely just as useful to you if you are doing quality benchmarks, target quality, or something else where SSIMULACRA2's correlation to subjective human ratings is more important than the exactness of the implementation to the reference.
Thank you for clarifing this, it was a misread on my side.
The overall percentage deviation from the reference implementation is marginal,
but just the pure existance of 'validate.py' looked to me like it must match.
You can build the image yourself, but have to switch off some packages or features - otherwise the image (linux-kernel + tools) is just too large or consumes too much memory. The original router has 8 megabytes RAM-memory and 2 Megabytes flash ("storage"). You can boot a recent kernel 6.16.5, but with 8mb there is not much left to work with 8-)
What seems missing: I can not see the answer from the different models. One have to rely on the "correctness" score.
Another minor thing: the scoring seems hardcoded to: 50% correctness, 30% cost, 20% latency - which is OK, but in my case i care more about correctness and latency I don't care.
Wow! This was my testprompt: