conv = lambda data:{ 'a': [el['a'] for el in data ],
'a_sum' : sum( [el['a'] for el in data ]),
'b': list(set( [el['b'] for el in data ])), }
conv(input_data)
which appears to have the same functionality.
This is quite off putting and it took me a while to dig down to find why convtools can offer more than just an extra abstraction layer to learn.
Perhaps pick an example that shows off the non trivial functions like joins or GroupBy?
Just to add my previous answer:
the trivial native python equivalent doesn't have the same functionality, because it consumes data iterator 3 times in your case, while convtools would consume it only once.
Similar to this [0] from a few years ago.
That one also had some basic encryption included.
Some previous discussions[1,2] of that tool highlights that this can easily be abused for xss or distribution of illegal materials which you may not want to find yourself the focus of.
Unsure I really understand the XSS risk that's discussed in the links you provided. Nobody cares about that origin because it has nothing of value hosted on it and it's not like you'd be able to access cookies from a different origin.
The worst you could do is exploit a browser zero-day, but you can do that on any static hosting site already!
First, it's not hard to imagine that someone might try to get their account banned for a GitHub terms of service violation keeping in mind that GitHub holds the account owner accountable for content in their repository. This is true even if that content is from other account holders they've given access to their repository. In this case, anonymous access is intentionally being provided which could of course go very, very, very wrong.
"You agree that you will not under any circumstances upload, post, host, or transmit any content that:
is unlawful or promotes unlawful activities;
is or contains sexually obscene content;
is libelous, defamatory, or fraudulent;
is discriminatory or abusive toward any individual or group;
gratuitously depicts or glorifies violence, including violent images;
contains or installs any active malware or exploits, or uses our platform for exploit delivery (such as part of a command and control system); or
infringes on any proprietary right of any party, including patent, trademark, trade secret, copyright, right of publicity, or other rights."
Understanding what the tool does, GitHub might be forgiving on the ToS violation front. The problem is with the second scenario: law enforcement. It's very likely that in a lot of jurisdictions, law enforcement, prosecutors, etc., wouldn't initially understand what's going on here and even if it can be explained to their satisfaction, I think very few of us would like to spend a night (or more) in jail while attempting to explain.
You are abusing trust - now it's going to be the jstrieb.github.io who is serving malware, and since his system serves whatever JS I provide by design it becomes a very effective XSS host.
It's not really jstrieb.github.io that's serving it, because since the content is in the url fragment, it is never sent to or from the server, it's handled entirely clientside.
>a very effective XSS host.
It can only do XSS against jstrieb.github.io which has nothing valuable. So it's not useful for anything. It can't be used in a <script> tag to obfuscate XSS attacks against other websites either, because the response isn't formatted as javascript. I guess it could be used in <iframes> on other websites in order to add obfuscation, but I think the use to attackers would be quite low.
You still need something to serve you the initial document.write js, unless you are going to convince people to open your links with locally saved "index.html". I called it "XSS" because you can execute arbitrary javascript, and I was trying to avoid bluntly calling it "malware".
Though I probably should have. Here is an example of a HackerNews login page served with jstrieb.github.com https://tinyurl.com/yypvh3by, you can login to news.ycombinator.com with it, but it easily could have been a phishing site.
My point is, this is a very good idea for offensive operations.
But someone could register the github account newsycombinator and then serve an identical phishing page at newsycombinator.github.io .
I guess you're right that it's useful for takedown resistance in phishing attacks. It's useless for small, sophisticated, targeted phishing attacks, but for large blunt untargeted phishing attacks it could be useful to have a site that would be difficult to take down and censor.
I like the idea, I've even written something similar myself, but for me a few things were not clear.
How would I load in data that is stored on my harddrive (not web addressable) without having to run my own server or go through the file selector popup box each time. User JS can't just read the harddisk (a requirement for web security). If I have to run my own server then a major selling point of this is gone.
I work with large tabled data and very often use vectorised functions using numpy or pandas, working on entire columns with a fantastically simple interface. A = B * x, for arrays B, scalars x returning a new array A. Is there something as beautiful available, working around the lack of operator overloading in JS? I've only seen string abuse like p('A') = p('B*x'). Auh.
Good question! Mike wrote a notebook about getting data into Observable here: https://beta.observablehq.com/@mbostock/introduction-to-data - the gist is that right now any host that supports CORS does the trick, and the combination of GitHub Gist & RawGit is pretty good for the task. It'd be nice to drag & drop data right in the app, though, so... stay tuned.
And, yep - something like pandas for JavaScript would also be nice... also stay tuned.
Yes, because "a" is not an object, it is a reference to an object. This reference is passed by value.
This kind of thing is why I'd recommand everybody to know C: when you know pointers there's no magic anywhere anymore
Something like this, without any kind of error checking being done.
#include <stdlib.h>
// assume some hash table library, exercise for the reader
typedef struct{} *hashtable_t;
extern hashtable_t hashtable_init(void);
extern void hashtable_put(hashtable_t hashtable, const char* key, const char* value);
extern void hashtable_dump(hashtable_t hashtable);
typedef struct {
hashtable_t table;
} data_t;
void foo(data_t* a)
{
hashtable_put(a->table, "otherKey", "b");
}
int main(void)
{
// let a = {key: 'a'};
data_t* a = (data_t*) malloc(sizeof(data_t));
a->table = hashtable_init();
hashtable_put(a->table, "key", "a");
foo(a);
// console.log( a );
hashtable_dump(a->table);
return 0;
}
The function foo() gets the numeric value of the a pointer, thus a parameter inside foo() points to the same memory location.
If C had pass-by-reference, it would be possible to give (implicitly) the memory location of the local variable a in main() instead. For example, like in Pascal (var) or C++ (& in function declaration).
There is another solution for having citations and references in figures: use Inkscape as suggested in the OP. In particular the save to pdf+latex feature. Any text elements in the figure are generated using latex. This means the correct font and the ability to write any latex you want as text. I still like hand editing figures to add complex annotations so I like being able to use Inkscape's GUI.
I just want to chime in and say that I've had a look and a quick play with mypost.io and think it looks awesome. Like many of the links on this HN thread, but with way more feature.
As you are allowing HTML entry, how are you protecting against Javascript inside user created pages?
I appreciate your great feedback. I barely had any influence when writing it, but all I knew was that I wanted to be able to write HTML and CSS -- and help my visitors with learning the basics, without having to even sign up for an account. All HTML is allowed though iframes seem to be so/so -- sometimes they work, sometimes not.
There is certain Javascript that is allowed, but others I've had to disallow. The way I've been able to allow it and monitor it is through BBCode. Basically writing [script src=myjsfile.js][/script] or something like [script]alert('hey there');[/script] is how you can get Javascript on a post but it does go through a filter beforehand to scan for any potential harmful code. Trying to write javascript the original way doesn't usually work.
At one point, I had Javascript as a main feature in the Advanced Options section, but I've since removed it, as I saw its potential use for abuse. As more and more users are using it from around the world, it's kind of interesting to see the clever "hack codes" people come up with. For the most part, I've mostly filtered out all "dangerous" code, but it still arises occasionally as the Internet evolves and more people are clever.
Mainly with the hacks I've seen: people using it to redirect directly to their own spammy websites. People creating divs that block MyPost but show a message on top and you can't do anything but go to the person's website or link. I've been able to filter these types of codes and prevent them from being entered into a post. The thing with Javascript though, there are dozens of ways to write the same code.
I even had to build my own captcha as people learned how to automate the creation of mass-posts. Some Russian guy emailed me (it was in Russian, but the translator basically told me he was pissed off because I added the captcha), but I knew it was him who created about 2000 posts in less than an hour. They then learned how to somehow bypass the Google Recaptcha and so I ended up building my own, which fortunately, at the moment, has successfully stopped bots from being able to automatically post thousands of posts at a time.
I'd rather people use it for its true purpose: getting webpages up on the Internet in seconds; learning how easy it is to code, etc. than to have hundreds of thousands of "spam" posts on the website. So those have been my battles since creating it: fighting bots and fighting people who are coming up with clever ways to "hack it".
I think it would be interesting to read about the decision process that decided on the 1.4MW of power generation to 6MWh of storage ratio. This is around 1:4.2h.
Is this ratio very specific to Ta’u island, with their usage patterns, or can this ratio be used as a guide for future installations?
I'm guessing that Ta'u has not as much technology that stays on overnight, and a more realistic ratio would be 1:6h or more.
If you are plotting using a canvas - not building a SVG or on the dom - then even 50000 points is fine, no need to filter even on mobile devices.
A 100,000 data point file would be about 500kb transfered to the client. This is still a tiny amount of data to play around with in JS. It is possible to build an array of image tiles, and zoom around like google maps. But keeping the image tiles in memory could be worse than just replotting if the plot function is fast and optimized enough.
