
Launch HN: Thematic (YC S17) Customer Feedback Analysis via NLP - zelandiya
Hi! I’m the CEO of Thematic, <a href="http:&#x2F;&#x2F;www.getthematic.com" rel="nofollow">http:&#x2F;&#x2F;www.getthematic.com</a>. We analyse customer feedback to tell companies how to increase customer satisfaction and reduce churn.<p>We are one of the handful of companies that got into YC through the Startup School, and (I have to say) the only company that signed YC itself as a customer!<p>I have a PhD in NLP and ML and was consulting when two large media companies came to me with a problem: They collect tons of customer feedback in free text as part of their NPS surveys, but don’t have the time to sift through the responses.<p>This turned out to be common. Most companies collect feedback but, especially in large companies, nobody reads this data, and definitely not people who are in charge of strategy. Customers are screaming what’s wrong and what they want, but nobody is listening.<p>I tried a few open-source packages but found that none worked well. Developed on canonical text like news article or Wikipedia, they either failed to understand the variety of expressions, or were too hard to explain. I wrote a new approach capitalising on my PhD and new Deep Learning approaches. It&#x27;s completely unsupervised: just needs raw data but, unlike topic modelling, produces clear and specific themes. My husband Nathan joined as a co-founder and for the next year we learned how to solve this problem in a way customer insights professionals find valuable.<p>Those media companies became customers and we quickly bootstrapped into a profitable startup. This is when Nathan signed up for YC’s Startup School. We grew 20% in those 10 weeks, loved the accountability and the focus. Our mentor suggested we apply for YC, which seemed like a crazy idea, but we gave it a go.<p>Fast-forward another 2 months, and we are just before Demo Day! Thematic grew 3x in that time, and we are working with brands like Vodafone, Air New Zealand, Stripe, Ableton, and Manpower Group.<p>Hope you found our story interesting, and happy to answer any questions.
======
textient
Suggestions to your business growth:

Surveys are the bread and butter for many market research companies. Most of
the corporates / enterprises typically engage with smaller to larger (and many
a times multiple) MR agencies. These MR companies can benefit from your
service. To explore these companies, you can check out the MR members
directory list from ESOMAR ,the voice of the MR (www.esomar.org), perhaps a
membership / participating in their events may help you. Other site is
agencyspotter.com

Explore publishing an article at the Greenbook blog run by Leonard Murphy
which is very relevant to this case and he also runs the IIEX events globally
( [http://iiex-na.insightinnovation.org/](http://iiex-
na.insightinnovation.org/) ) where tools /services such as yours are very much
the hot thing..

Check out Unilever Foundry
[https://foundry.unilever.com/](https://foundry.unilever.com/) . You can sign
up and explore if you can help solve some of their problems with your
solution. They select and fund Pilot projects

Twitter hashtags to get your tool noted in the MR industry #mrx #newmr

Ads / promotions: check out [http://newmr.org/](http://newmr.org/) ,
[http://www.greenbookblog.org/](http://www.greenbookblog.org/)

Explore if you have complementary business synergies to present with
[https://www.zappistore.com/](https://www.zappistore.com/) (e.g your product
could be a part of the Zappistore platform as an App.)

Best Wishes, N.Sankar
[https://www.linkedin.com/in/nsk007](https://www.linkedin.com/in/nsk007)

~~~
zelandiya
Wow, this is super helpful, thanks! I will check out these resources.

------
wadkar
Congratulations and best wishes!

I for one really liked the demo and the blog - specifically, (a) I have great
exemplars for what you mean by "theme", and (b) this post[1] shows great
insights into your thinking about the problem faced by your customers

> Developed on canonical text like news article or Wikipedia, they either
> failed to understand the variety of expressions, or were too hard to
> explain.

It appears to me that the current methods and resulting tools are heavily
dependent on the problem formulation (or domain in general). Moreover, no
matter how fancy your technique is (or "how deep is your net"), the resulting
model won't work unless you take specific steps to train it on data from the
domain.

Yes, what I just said sounds borderline truism. However, I am more interested
in discussing why it is so? Here's my initial thinking:

Let us look at (one of) the definition of Machine Learning, from Prof Tom
Mitchell's textbook, "A computer program is said to learn from experience E
with respect to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with experience E."

Here, experience E can be loosely considered as the amount of data you have
for training - obviously, more data (i.e. training) should improve learning.
However, the abstraction of T and P hides an important underlying problem of
specification - or in other words, formulation of T (and E).

Thoughts?

> I wrote a new approach [capitalizing] on my PhD and new Deep Learning
> approaches.

I hope we get to see some of your insights in a paper or article (or blog post
:)

[1] [https://www.getthematic.com/post/visualizing-customer-
feedba...](https://www.getthematic.com/post/visualizing-customer-feedback-
three-alternatives-word-clouds/)

~~~
zelandiya
Thanks for your thoughts! Definitely sounds like an interesting thought to
explore.

I like to think about ML in terms of how children learn language: through
observation in their environment. The training data is a simulation of that
environment.

Glad you liked the website and the blog post!

------
randoman
I really enjoyed reading about your company on Idealog
([https://idealog.co.nz/technologymonth/2017/08/meet-
thematic-...](https://idealog.co.nz/technologymonth/2017/08/meet-thematic-new-
zealand-startup-making-waves-y-combinator)). It was especially great to see
how you bootstrapped and found product-market fit rather than just raising a
bunch of cash for an idea that may or may not be applicable in the real world.
Congratulations on the launch and your success with YC. Go Kiwis!

~~~
zelandiya
Thanks so much! Appreciate the kind words. :)

------
tixocloud
Congratulations on your launch! This is very interesting and it's something
that has great applicability. Being in financial services, we collect a ton of
feedback and audio but not enough manpower to process it all. I love the
themed approach but am curious about the subthemes. Would you be able to shed
any light on that?

I'm also happy to make introductions if you're ever thinking about expanding
up north to Canada.

~~~
zelandiya
Subthemes are basically more specific versions of the themes. Check out an
example on our Twitch demo: [https://demo-
twitch.getthematic.com/#!/demo/main/dashboard/2...](https://demo-
twitch.getthematic.com/#!/demo/main/dashboard/2/visualization/1) You will see
a base theme "use chat", with subthemes like "chat in landscape mode", "chat
on a tablet", "type in chat" etc. Does this help?

Thank you so much for the offer of introductions. Definitely interested, as we
sell internationally, not just in the US.

~~~
tixocloud
Thanks. Would subthemes almost/always fall under the context of the
overarching theme?

------
jstandard
Great tool, this is an area I've been looking forward to more automation in.

I might have missed it on the website, how does pricing work?

Also, do you have any integrations with other tools like Intercom or Zendesk
to ease data-sharing? A monthly insights report generated directly off of my
main customer support tool can replace hours of manual work.

~~~
zelandiya
Thank you! Our pricing is custom, it's a subscription model. A monthly report
is quite a common use case. We have a Zendesk but not yet an Intercom
integration. Happy to talk further, please feel free to request a demo via our
website getthematic.com

------
Yertis
Love this! When I was at one of the big tech companies years ago, we tried to
do something like this for reviews for all the products, and see how they
stacked up in the marketplace. Was a really challenging problem to do in an
automated way at scale -- definitely wish we had a solution like this!

Do you guys see yourselves sticking to a model that spits out analysis, and
let customers decide what insights to gain from the data? Or could there be a
path where eventually it lets users take specific actions based on the data?

~~~
zelandiya
This is a really interesting point. I think what you mean is operationalising
the analysis. We let customers decide what those actions are and then provide
integration to help this happen. For example, one customer has a call center
that receives an email from Thematic each morning stating which customers are
likely to churn based on themes they mention. They call them up first.

~~~
Yertis
Ah, that's awesome -- the point on integration especially is a big one.

The more ways clients can hook the insights into their CRMs/workflow/user
base, the more they can "operationalize it", as you eloquently put it, and
make it a part of their workflows, the way the client you described does with
their call center. I love it!

------
seanwilson
How do you go about evaluating the accuracy of the themes and action items? Do
you create a test set with obvious themes and actions and check the results
for example?

~~~
zelandiya
Good question! Accuracy of themes is subjective. We run several studies where
we used opinions of several people on what the themes should be and then
compared the overlap between people with the overlap Thematic has with each of
them. This is also referred to as "indexing consistency" in Information
Science.

You can check specific examples in our white paper:
[http://www.getthematic.com/net-promoter-score-
verbatims/](http://www.getthematic.com/net-promoter-score-verbatims/) (Don't
need to download it, the bulk of the results is published on the page.)

In practice, some customers have read samples of responses and expect certain
themes. Others have manually tagged them in the past. They compare their
results with their fundings and can see straight away if our results are
accurate and reliable.

------
forgotmysn
would you say that your solution provides a sentiment analysis, similar to
Quid, or do you focus on action items and things that product managers can
actually address?

~~~
nshgraph
We do provide sentiment but the main benefit is definitely the themes which
form the action items that can be addressed.

We show what areas to improve, not just how people feel (although that is
important in deciding)

~~~
alexcnwy
Great product and congratz on your success so far!

Would you mind please elaborating a little more on how you're thinking about
theme-specific sentiment (for the NLP students here :) - is it along the lines
of Socher's aspect-specific sentiment or something else?

------
AznHisoka
I saw there were samples but is this similar to how BloomBerry comes up with
"topics" for popular questions for a keyword? They use some sort of NLP
extraction of noun/verb phrases.

Example: Most popular topics/themes related to Vodafone:
[https://app.bloomberry.com/questions;q=vodafone](https://app.bloomberry.com/questions;q=vodafone)

~~~
zelandiya
Thanks for pointing out! Bloomberry has some useful themes, yes, it does look
similar. Good to know they exist.

------
pj_mukh
This is super cool, I also appreciate the demo's you have on your website
(airlines, MBA schools etc.). Makes the end result super clear.

I don't know much about NLP but are you _only_ using unsupervised learning on
the raw data? I would think you would need an NLP layer as well that sorts out
basic synonymical issues, phrasing differences etc.?

~~~
zelandiya
Thank you! No training data is required to use Thematic and customers don't
need to tell us what they want to find in the data. Hence we say
"unsupervised". And yes, we extract synonyms and paraphrases from raw data.

However, we do have tools to review and adjust the results of Thematic by
hand. For example, we have an internal drag and drop interface. Some customers
really like to change the themes based on their view of the data. But it also
helps to remove any inaccuracies, e.g. an incorrectly merged theme.

------
bberenberg
Are you planning to provide your service as an API to other companies that
want to wrap a product around your work?

~~~
zelandiya
No, not currently, but maybe in the future.

------
bitL
Congrats and good luck!

I work on the same, just for my own company to automate customer interaction
(well, at least 99% of it).

~~~
zelandiya
Thank you! Yes, there are many applications beyond customer insights. We are
dabbling into these also. What's your company?

------
hobolord
Congrats on the launch! I think I saw you talk recently in NZ, sounds like you
have an exciting path forward

~~~
zelandiya
Cheers!

------
0xbb
You guys hiring? I'm not a PhD or anything, just an undergrad with an interest
in NLP.

