
Mozilla launches voice search, file-sharing and note-taking tools for Firefox - denchikceo
https://techcrunch.com/2017/08/01/mozilla-launches-experimental-voice-search-file-sharing-and-note-taking-tools-for-firefox/
======
albertzeyer
Some more information in the official blog post:
[https://blog.mozilla.org/blog/2017/08/01/new-test-pilot-
expe...](https://blog.mozilla.org/blog/2017/08/01/new-test-pilot-experiments/)

I was especially interested in the Voice Fill (speech recognition) technology.
Landing page: [https://testpilot.firefox.com/experiments/voice-
fill](https://testpilot.firefox.com/experiments/voice-fill)

It seems the project is here:
[https://github.com/mozilla/speaktome/](https://github.com/mozilla/speaktome/)

This seems as if it actually is a webservice. From the code
([https://github.com/mozilla/speaktome/blob/master/extension/c...](https://github.com/mozilla/speaktome/blob/master/extension/content.js)),
I see: const STT_SERVER_URL =
"[https://speaktome.services.mozilla.com";](https://speaktome.services.mozilla.com";)

Actually, I think this can be very easily done fully client-side, with good
accuracy. Even on Android, the voice recognition can run client-side /
offline.

I wonder if the project is in any way related to their DeepSpeech project
([https://github.com/mozilla/DeepSpeech](https://github.com/mozilla/DeepSpeech)).
Maybe they use DeepSpeech on the server-side? At some other place they call it
Pipsqueak, not sure if this is yet something else.

And maybe also related is their common voice project
([https://voice.mozilla.org/](https://voice.mozilla.org/)). Recent discussion
here on HN
([https://news.ycombinator.com/item?id=14794654](https://news.ycombinator.com/item?id=14794654)).

Some more information also here: [https://research.mozilla.org/machine-
learning/](https://research.mozilla.org/machine-learning/)

~~~
fabrice_d
"this can be very easily done fully client-side" : maybe, if you have the
voice model _and_ and inference engine that runs well on devices. Mozilla
doesn't have that yet, so this experiment uses a backend running a Kaldi
server and model that uses too much memory to run locally.

Once DeepSpeech is ready I'm pretty sure they will switch to that, and
ultimately to on-device voice recognition with PipSqueak (PipSqueak is
expected to be an inference engine usable on devices). Unfortunately none of
these projects are far along enough to be usable.

Common Voice is mostly related to DeepSpeech as this will help getting data to
train the engine.

------
jacquesm
Browsers should be browsers, not a way to get a whole bunch of trojan horse
software installed on your machine.

If I wanted voice search, file sharing and/or note taking I would find the
appropriate piece of software for that task and install it separately.

All this does is increasing the attack surface of the browser in a ridiculous
manner providing features that only very few will use. File-sharing? What
could possibly go wrong?

And in the meantime Thunderbird, a stand-alone program that had a decent
following got the axe in favor of more browser bloat.

I really no longer understand Mozilla's mission.

~~~
482794793792894
The title is misleading. These are not part of Firefox. They are in the Test
Pilot program, which is a way for Mozilla to gather feedback on what features
they should include, how they should change them if they do include them and
all in all is completely open-ended. They might also release these as
extensions or somesuch. They are all coded as WebExtensions (except for that
file-sharing thing, that's just a webpage), so that's very well a possibility.

Also, browsers already have code to record from your microphone, due to the
WebRTC web standard, so it's not like this opens an entire new class of
potential vulnerabilities.

~~~
coldtea
> _The title is misleading. These are not part of Firefox._

Doesn't matter. They shouldn't be part of Mozilla's output either...

------
notheguyouthink
I just wish it was .. faster.

Lately I've been moving away from Google everywhere I can. I moved everything
but Google Voice. Yes, even Google Search - I've moved to DuckDuckGo. On
windows however, I had to fall back to Chrome, because I was just shocked at
how slow Firefox was.

Opening pages like Twitch.tv proved to be shockingly slow. Furthermore, my
habit of opening many tabs in the background like I do in Chrome/Safari was
massively slower in Firefox because while Chrome doesn't autoplay new-hidden
tabs, Firefox does - I imagine Chrome feels faster there because it's not
running nearly as much stuff at once.

Pretty much everything of Firefox felt slower for me. And this is from someone
that _really wants to get away from Chrome!_ On OS X, I've long switched to
Safari and DuckDuckGo, and been quite happy. I've had zero complaints about
performance with Safari.

So.. I don't know what they need to do, but I'm really hoping they do
something.

~~~
artursapek
It doesn't help that they keep cramming features into Firefox. Chrome feels
pretty bare in comparison.

~~~
pcwalton
Chrome has consisted of significantly (i.e. millions of lines) more code than
Firefox has for quite a while now.

~~~
H4CK3RM4N
I feel like I heard that a Chrome install was comparable in size to an average
Linux install a year or two ago.

------
dmitriid
Didn't they say not a few months ago that "the reason Firefox has fallen
behind is that we focused on stuff that wasn't browser-related like MozillaOS
etc."?

Learning from own mistakes anyone?

~~~
yAnonymous
Seriously. They're behind in speed, security, quality of the developer tools
etc. and introduce new useless toy features and one re-design after the other.

I appreciate a lot of stuff Mozilla do outside the browser for the open
internet, but the development of Firefox is pretty bad and it looks like the
project leaders are incompetent.

Firefox is a sad example of how a successful software can be overtaken by a
new competitor within only a few years due to a lack of focus.

~~~
syrrim
>due to a lack of focus.

And due to their competitors ability to advertise on the majority of people's
home page. For most people, chrome wasn't a firefox alternative, it was an
internet explorer alternative.

~~~
yAnonymous
I'd like to know how much of an impact that had, but nonetheless Chrome is
technically in many areas the best browser today and Mozilla, being the clear
market leader at one point, could have done a lot more to prevent that.

Firefox was in need of technical improvements for some time, but they only
started acting when Chrome already had a noticeable market share and that's
not how you stay at the top.

------
woranl
Mozilla should focus on building the browser instead of building app that
competes with other developers. They need to start listening to the developers
community and stop being arrogant and ignorant. You build the foundation, and
developers build the app.

~~~
SwellJoe
That's a weird take. Mozilla has a wide variety of developers, working on all
sorts of things. One of those things is experiments to figure out what the
browser "foundation" should actually include.

I like that they're experimenting to figure out what good usability looks like
and what the next generation of "browser" should deliver. Some things should
just be there when you want them...not an app you install separately or a
website you go to for the functionality. I don't know if all of these new
things fit that category (voice stuff probably yes, notes probably yes, large
file sharing maybe?), but I know I want Mozilla to keep trying new things.

Also, if Mozillians aren't building user-facing things with the "foundation",
they don't actually know what the foundation should look like for developers.
You have to be a developer using the platform daily in order to understand
what it does right and wrong. I don't know how they could do that without
actually building things that sit atop the platform.

~~~
woranl
I just found it damaging to their vision and brand. Take Hello and Persona as
example. User focused? Yes. Failure? Absolutely. They need to be developers
focused. Less and less developers are testing their apps with Firefox as the
primary platform.

~~~
bobajeff
That's becuase of their declining marketshare (which is mostly due to
strategic disadvantages in distribution channels and marketing budget) not
their brand or split focus.

~~~
woranl
Budget has always been their disadvantages, yet in the pre-Chrome days, they
were core product focused and won many developers support. Their vision was
once inspiring and admirable. This was how they won over IE. However, ever
since Chrome was introduced, most of their innovations are either catch up or
gimmicky features - not to mention the political turmoil/non sense. I really
wish they can focus on their core product again.

~~~
bobajeff
They got kind of lucky too. Their main competition abandoned development of
their browser for about 4 years. Now, there are four browsers in active
development each regularly communicating with web developers.

Given their pick it's obvious web developers would rather focus on the
browsers with the most share of users.

------
JepZ
Sorry, but I will not use any speech recognition service until it becomes a
pure on-device service.

------
romanovcode
I hope Firefox succeeds and destroys Chrome.

~~~
ece
After the 54 release, it is the better browser frankly, low latency, and not a
resource hog. Plugins, extensions, privacy and security defaults all seem to
be better as well.

Using firefox focus on android is pretty great too. The normal firefox on
android is just slow, not sure why.

~~~
randie63
because its webview which is just chromium. They don't use the gecko engine
like in desktop firefox

~~~
thristian
I believe Firefox Focus (on Android and iOS) uses the OS native web view
(effectively, Chrome or Safari), but the actual Firefox for Android uses
Gecko.

~~~
ece
^ This seems right from my quick googling, there is this project for a gecko
webview, which needs work:
[https://wiki.mozilla.org/Mobile/GeckoView](https://wiki.mozilla.org/Mobile/GeckoView)
and mentions firefox focus' gecko build option.

Firefox on android is too slow to use a lot of times. Something mozilla should
be able to do better.

~~~
toweringgoat
Mozilla just don't care about Firefox on Android, the naive observer would be
forgiven for thinking they're actively sabotaging it.

------
rudedogg
Slightly off-topic, but does anyone use a Vim plugin for Firefox? I keep
trying to switch, but the scrolling of VimFX drives me crazy. Here's an issue
outlining the behavior:
[https://github.com/akhodakivskiy/VimFx/issues/830](https://github.com/akhodakivskiy/VimFx/issues/830)

Are there any other Vim plugins without this issue?

~~~
thristian
Vimium-FF <[https://addons.mozilla.org/en-US/firefox/addon/vimium-
ff/>](https://addons.mozilla.org/en-US/firefox/addon/vimium-ff/>) is the only
Vim-keyboard-binding plugin I know that will work with Firefox 57 and above. I
haven't noticed any scroll jankiness as you describe, but it's possible I have
my key-repeat-delay set low enough that it doesn't affect me.

~~~
rudedogg
Gave this a try, and it works! No weird jump after the first scroll.

Also, for anyone using 1Password, they have a beta of a multiprocess
compatible FireFox extension that seems to work fine.

Thanks!

------
paulio
I find it hard to believe that speed is really the issue here. I suspect it's
more likely it's the aggressive advertising scheme in which Google push for
Chrome.

I think these projects which Mozilla test/launch are an indication that even
they don't know how to stop the eroding of Firefox market share.

If I were Mozilla I'd be seriously worried about Electron.

~~~
mirsadm
Speed is definitely an issue on the mobile app. Personally I think Mozilla
should try harder with the mobile market. Providing an ad and tracking
blocking browser is still desirable on Android. Firefox is just too slow at
the moment.

~~~
sornars
Doesn't Firefox focus fill that gap? [https://www.mozilla.org/en-
US/firefox/focus/](https://www.mozilla.org/en-US/firefox/focus/)

Super speedy as it uses an Android Webview and privacy focused.

------
bad_user
I wonder what happens to the previous Test Pilot experiments.

I loved the experimental home page and the Tab Center [1]. I really hope it
continues to live.

[1]
[https://github.com/bwinton/TabCenter](https://github.com/bwinton/TabCenter)

~~~
sp332
On the main [https://testpilot.firefox.com/](https://testpilot.firefox.com/)
page, scroll down and click "view past experiments". Each one gets a write-up
when it's over.

~~~
jgruen
Hey, we're a little behind on writing up the last few experiments
unfortunately. We'll be adding full reports over the next few weeks.

------
robfreudenreich
I don't know here Mozilla got their inspiration, but Send looks pretty similar
to our E2E encrypted file sharing app Whisply:
[https://whisp.ly](https://whisp.ly)

PS: Whisply even has more features and a detailed description how its
encryption works:
[https://whisp.ly/static/whisplyTechnicalOverview_20151201.pd...](https://whisp.ly/static/whisplyTechnicalOverview_20151201.pdf)

~~~
problems
There's many such tools available, RiseUp runs one called Up1. See
[https://share.riseup.net/](https://share.riseup.net/)

It's quite nice, has integrated image and video viewers as well as a pastebin
all end-to-end encrypted.

~~~
robfreudenreich
Up1 looks nice, but is internally using SJCL which is slow and does not
support bigger files ("This is not a problem with sub-10MB images"). Send -
and Whisply - are built on the new WebCrypto APIs which are faster and allow
bigger files, up to 1 GB in both cases.

------
r3bl
I've seen all the promo videos, and some questions remain unanswered:

1\. What's Send encrypting with? It doesn't show anything related to setting
up a password during the video.

2\. What's Notes syncing with?

3\. What's Voice using to analyze the voice commands?

~~~
wongarsu
>What's Send encrypting with? It doesn't show anything related to setting up a
password during the video.

Likely a random string that is added to the download URL as Fragment (the part
after # that doesn't get sent to the server).

The download URLs look like this:

[https://send.firefox.com/download/50d43ef5f3/#GqCJXOUnCyxYGc...](https://send.firefox.com/download/50d43ef5f3/#GqCJXOUnCyxYGc7amk7opQ)

I agree that there is very little explanation beyond "look at this cool
thing".

------
SubiculumCode
Wowsers. The Nightly version of Firefox is FAST.

------
SwellJoe
Somehow, I assumed "Notes" would be "notes for this page". But, it's just a
notepad...no awareness of where I am. That's much less interesting/useful than
I thought it would be.

Is there a browser plugin out there that provides a notes sidebar for the page
you're currently on, and allows flipping through the notes with a link to the
pages they relate to? Because that'd be awesome. Googling only reveals some
kitschy "post-it on a web page" type note things, which is definitely not what
I want.

~~~
6ak74rfy
Look at Hypothesis, which does something better : it allows you to annotate
your web pages.

~~~
Vinnl
Note that Hypothes.is notes are public.

------
mynewtb
Opera 12 called and it's rotating in its grave. Unite was such an amazing
feature. And notes were available in Opera even before I started using it.
Opera, I miss you, it still hurts.

------
Dirlewanger
I wish they'd stop focusing on the flavor-of-the-week technology gimmick
bullshit and focus on making Firefox the better performing browser. Chrome
outperforms it in nearly every way by margins that grow with every release. My
reasons for sticking with FF grow fewer every day.

~~~
ancarda
Are Mozilla paying for people to do these experiments or is it people sending
patches? I don't know but I'd imagine a lot of it is the latter.

Have you tried a recent Nightly? Firefox is getting faster:
[http://www.techradar.com/news/firefoxs-blazing-speed-with-
hu...](http://www.techradar.com/news/firefoxs-blazing-speed-with-huge-numbers-
of-tabs-leaves-chrome-in-the-dust) [1]. As Mozilla continues to pull in code
from Servo and implement e10s, Fiefox will perform better and crash less.

[1] discussion:
[https://news.ycombinator.com/item?id=14848836](https://news.ycombinator.com/item?id=14848836)

~~~
Sylos
> Are Mozilla paying for people to do these experiments or is it people
> sending patches? I don't know but I'd imagine a lot of it is the latter.

As far as I'm aware, it is mostly developed by Mozilla employees, but heavily
used for new devs to get their feet wet. Not only is it not mission-critical
when they make mistakes there, they also get a ton of feedback on their work.

------
jeshwanth
Voice fill is awesome feature. Kudos to Mozilla :)

------
quanticle
Is Mozilla committing publicly to supporting these new features for a
significant period of time? I've been slightly wary of integrating new Firefox
features into my workflow ever since they cut off support for Firefox Hello in
Firefox 49.

~~~
Sylos
No, not at all. These are in the Test Pilot program, which is specifically for
trying new things and getting feedback if and how they should be included in
Firefox.

So, most of these are not here to stay for the long term.

They might be included, they might be published as an extension either
officially or just forked by someone, but overall, no guarantees on anything.

------
sysdyne
When is the end of life for these features?

~~~
Sylos
These Test Pilot experiments usually last for a few months, but there's no
fixed end date, I don't think. It's mostly just a matter of having gathered
enough feedback.

But well, if it seems like a feature that people want, then it gets included
into Firefox itself and stays for longer.

Then, again, I for example can't at all imagine them including that file-
sending mechanism. I doubt they even have the server capacity for that...

------
nkkollaw
Looks like a good waste of resources to implement things people don't want in
a browser anyway, in a desperate attempt to get people to consider using
Firefox again.

If I was Mozilla I'd allocate programming time to catch up with Chrome now
that they don't have the phone OS thing anymore, instead of creating more
distractions.

They really blew it. I remember a few years ago when I used both Firefox and
Thunderbird as my default browser/email client...

------
mrspeaker
I feel like I must be reaching the "get off my lawn" phase of my life, because
I can't even comprehend a situation where I'd ever want to talk at my
browser... maybe when I'm at home alone and, couldn't type for some reason?
Certainly not in the office. Am I missing a use case, or am I just old now?

~~~
sp332
Lots of people are faster at speaking than typing. Or maybe you're reading
from a document (or pill bottle) and don't want to type with one hand or
whatever. But what I usually use it for is if I'm having a conversation with
someone and we want to look something up. It feels more anti-social to type
silently for a while than to make the computer part of the conversation.

~~~
mrspeaker
But it's _only_ for searching after you've already navigated to a search
engine, right? You'd have to be extremely slow at peck-and-hunting for it to
save you much time (especially as search strings are so short) - and in the
video you have to click "submit" to even do the search (and then presumably
navigate normally with mouse). I can't see many instances where this could be
worth it. Maybe I'm actually too young to see why it's useful!

~~~
burkaman
Maybe the search engine is your homepage, or you just have to hit "g" and
enter to get there, or it's bookmarked. Maybe you don't have hands, but you
have an easy way to move a mouse around and click. Maybe you're eating a
sandwich with one hand and don't feel like putting it down. Maybe you won't
use it for a short query like "weather", but you will for sentence-long
queries. Maybe you want to search "linux won't recognize wireless keyboard".

It's very easy to think of instances where this will be helpful.

------
15charlimit
But why?

It's a browser. All it should do (and do well) is display content.

I don't want a bunch of extra garbage tossed in because it sounds good on some
marketing slide.

Chrome has been and continues to eat FF's marketshare alive because it has
been both faster and lighter. More junk is not going to help FF beat them.

~~~
newscracker
> Chrome has been and continues to eat FF's marketshare alive because it has
> been both faster and lighter.

I believe Chrome got a larger market share also because it had the giant force
of Google marketing it heavily on its properties for years, whereas Firefox
only had some support of that sort from the Yahoo search deal (IIRC). More
people land on Google's online properties than on Yahoo's.

I personally don't agree that Chrome is faster and lighter. For me (as a heavy
multi-tab user) it has always been slow as molasses and a laggy browser.

------
retox
I just want to browse the web!

~~~
Certhas
What's stopping you? Optional experimental plug ins?

------
hexmiles
what will happen to test pilot after the non-webextension are not allowed?

As far i know must of these feature are implemented as extension and i don't
thing webextension have the api to do a lot of thing that are in test pilot.

~~~
jgruen
We're moving to a webExtensiony future. The idea here is that we'll be a first
party consumer of the webExtension APIs so we can help test/drive/expand
capabilities.

------
jug
I don’t need any of this in a browser, and if I did, it feels like extension
territory. Weren’t Mozilla over this? Here I am, looking forward to Project
Quantum technologies.

------
petre
What would be useful in FF is a content blocker, especially in the mobile
version. Not pocket, file sharing apps, voice, notes. There are web apps for
that.

~~~
Sylos
Go into about:config and set "privacy.trackingprotection.enabled" to true.
This is default-enabled in Private Browsing.

Also, uBlock Origin and similar are perfectly functional extensions. Takes
just three clicks to install uBlock Origin on a fresh desktop Firefox
installation.

If you're wanting to tell me that this content blocker should be default-
enabled, good luck explaining to webpage owners that they won't make ad money
off of Firefox anymore, but should still support it.

~~~
petre
I already have uBlock Origin on Firefox on the desktop. It shoudn't be enabled
by default but a content blocker would be an useful addition to a browser
rather than notes, voice, pocket, chat and who knows what lass than useful
feature they might think of next.

------
dijit
I was interested in Send until I realised it wasn't p2p.

:(

------
visarga
I only want a sweet WaveNet-based TTS voice. Baidu's DeepVoice seems
promising.

------
erikb
I don't know about other users but I use FF less and less. I don't need all
this additional stuff. And it was painful how we lost features like the
firebug plugin.

It's a pity considering a few years back I was the greatest FF fan.

------
eridius
send.firefox.com says that my browser (Safari) is unsupported. Anyone know
what particular "web technology" the site uses that Safari doesn't support?

~~~
sp332
This is the only one I can think of, not sure why that site would require it
though. [https://caniuse.com/#feat=input-file-
directory](https://caniuse.com/#feat=input-file-directory)

~~~
eridius
Found it. It's checking for window.crypto.subtle. Looks like Safari TP
supports this. I believe the problem with Safari 10 is that it implemented an
older version of the web cryptography standard.

~~~
jgruen
yep

------
sunseb
This browser is more and more bloated... :-(

~~~
SubiculumCode
These featured come in an optional extension and your comment is fud.

------
znpy
Mozilla is doing everything but what is supposed to do:make Firefox blazing
fast.

~~~
toyg
They actually did, the recent switch to multiprocessing made a massive
difference. I've been happily using Vivaldi for a few months now, but I'm
tempted to go back to FF as the speed gap has been wiped out of late - only
cold startups are a bit slower, actual pageload is absolutely on par or
better. This on Mac, at least, but I expect it will have been similar on other
platforms.

~~~
znpy
Last time I tried it still was slow... I'll give it another try.

~~~
tmbsundar
When was the last time you tried? R55 is way snappier than Chrome. Nightly is
also fast.

------
free2rhyme214
None of these will get me to switch. Chrome is still faster than Firefox,
sorry Mozilla.

------
cannonedhamster
Firefox is the browser I really want to like, but can't use. I recently tried
to switch back, I kept running into problems with webpages not working,
figured it must be an add-on, removed them all, still frozen pages persisted.
A web browser that chokes on pages frequently is dead in the water for me.

Then there's the lack of compatibility with their mobile browser for
extensions, so most of my extensions didn't work on Android tablets. It was
definitely faster than it had been the last time I used it, but at this point,
it needs to be significantly better than Chrome to win back users and it's not
there except for a small group of people.

I do feel that the Voice Fill is a step in the right direction. I'd prefer
that the notes application was a dedicated product with an add in that tightly
integrated with the browser, a webapp in a wrapper would be fine really.

