Why does every data science infrastructure provider want to be full stack? I really don't want to use an online IDE, I really don't want to be locked into one particular way of distributing workloads, I really don't want to move my code from GitHub. All of the individual components look great, but the all-or-nothing proposition is really off-putting.
When approaching a beachhead market, it's best to try to deliver a whole product rather than to spread one's self thin specializing for disparate consumers. Clearly users will appreciate some features of their offering more than others; shooting for full-stack at launch both gets the company more feedback and allows them the most control over the initial relationship.
I totally agree that the full-stack offering can be rather off-putting, but the business side of DS still doesn't understand how to reliably make money from this market. Cloudera (and friends) have made money from a full-stack offering, so it's easy try to imitate this success.
IMO the online IDE is very useful for expositions and small projects, but indeed a limiting playground for larger projects. I'd love to see SAAS/PAAS do better at co-locating their offerings so that, e.g., I could run part of the stack on my own hardware (which is where the data lives). Perhaps Docker would help facilitate this business model.
We definitely don't want to lock you in, just to round off the rough edges of the "on your own" experience. You can create a project from GitHub and push/pull to GitHub from Sense, and our distributed computing setup was chosen to be low-level enough to support many of the existing distributed computing tools like doRedis, but still way easier than provisioning and plumbing servers.
We know that there's room for improvement on some workflows, in particular Git interop, but we have our sights on a platform that plays well with as many others as possible. Please ping us with any other thoughts you may have!
Or do you want more utilities that work together and run locally? (i.e. repeatable workflows in project bundles of scripts + data + saved notebook logs etc).
We were just looking at it.
It's a nice interface -- it's logical and someone gave it a thought.
They also have some of the features we (at SageMathCloud) were talking about with regards
to socializing: e.g. following a user and in reverse having followers,
etc.
Also nice is that they have project templates (for python,
julia and R, basically), where you get some explanatory *.md files to
read. (I.e. not an empty project).
You can also start a project from a git/github source and so on.
It seems to be heavily broken right now, or heavily unstable. Hmpf...
I'm trying to start the ipython notebook (you have to do this on
demand), then you can start editing the files ... but it is stuck. Now
it's completely broken.
The terminal works fine now, and a project is running in some kind of
isolated container in beefy linux ubuntu 14.04, running under a common
"sense" user (which is the only user).
They are also running some node.js client and their "butterfly" server.
Ok, well, I tried to test it further or add a collaborator, but
the website just says "502 application not responding".
It didn't seem like they have real-time editing or LaTeX.
few more details. ... now it runs fine again. two screenshots
attached. ... one is the project view, the other one the "workbench".
They run a plain "R" terminal, but with a neat HTML output. It's a
vertically split editor, where the .r file is on the left, and you can
select lines of text which are then executed via "ctrl+return". That's
exactly how those R people work (instead of having cells like ipython
or sagews)
The same for python files: one or all lines are fed into the "ipython
terminal", not the notebook (!), and then translated to some html
output on the right. plots work fine, etc.
So, on the right of the screenshots, you can see the copies of the input lines.
By default, they add a .gitconfig file with the username and email
address. That's cool, and I guess this could be done for SMC as well.
There is no auto-save for files, just lost the content.
I took a look - registered for the free personal version to try it out. I like the ideas and a nice working demo that sucked me in. I fiddled around with modifying the R code and running it again.
It was a little confusing in places. Eventually I figured out that I need to save the modifications, before running it again. I think I ended up starting up a couple console sessions that I didn't really need. Eventually figured out they were open and that just navigating away from a session didn't shut that session down. I went back in and manually stopped them.
Things that resonate with me.
Running on a server in the cloud - including instances that are substantially more powerful than my laptop
Dead easy deployment (being a sysadmin or devops type admin is low on my priority list of things to learn)
Interesting demo for me to play with
Online collaboration - even though I mostly work solo
Encouragement to practice version control with research code
Scheduled jobs and automating pipelines
I will certainly keep an eye on how it develops and create a project of my own on there in order to better understand the features, what works and what is missing
Thanks for the hard work to get to this point
I suspect that I will continue to play around with it and post some of my own code
Thanks for the feedback. We're always thinking about how to improve the workflow and will take your comments into account when we next revise. Please ping us on intercom with any other thoughts!
Who do you think is your biggest competitor right now? Do you think that you'll be fighting with Microsoft's Azure Machine Learning offering for the same audience ? How do the two platform compare?
SAS is the huge incumbent, but open source tools like R and Spark and cloud-centric platforms are going to win, no question. We're already well into this transition and tools like Sense should accelerate it.
Just signed up to test things out. Really nice interface and great job with having the onboarding process although I did get slightly lost when it came to editing the R file. Will continue to test it out a little bit later.
Excited to see if this takes off - working on a startup where an integration could be beneficial to both sides. Just curious who you feel your target customer is?
The interface looks nice. How do you get data into the platform? The field I work in (genomics) usually uses a bunch of flat text/binary files instead of databases. How would this platform interface with that kind of data?
You can upload small to medium files directly into your project, which lives on a distributed filesystem. For larger data, we typically recommend pulling data directly from S3 or production databases.
