Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Trickle – Let GPT-4 Understand Your Screenshots (trickle.so)
20 points by jarodxu on Sept 8, 2023 | hide | past | favorite | 13 comments
Hey there HN!

It took nearly 5 years for my team and me to truly find right direction. So, after introducing our work, I'm keen to share the story behind it. The following content is divided into two parts. If you're not interested in the backstory of the product, feel free to skip the content after the divider.

> The Problem We're Solving: During a casual afternoon, while brainstorming what to do next on a WeWork sofa, we realized that almost everyone present had a habit of saving information via screenshots. When I opened my photo gallery, I was astonished to see that more than half were such screenshots. Given that traditional OCR and gallery apps hadn't really addressed our screenshot chaos, we decided to build something to solve our own problem.

> How it works: At first glance, you might think Trickle is a manual screenshot version of Rewind. But in reality, they're vastly different. All you need to do is send your screenshots, and let Trickle handle the rest:

[1] Trickle doesn't constantly record the entire desktop, so it won't consume all your Mac's storage or affect its performance. Moreover, it won't give you the unsettling feeling of being constantly watched.

[2] Although we have a Mac screenshot tool, and a browser extension is on its way, you don't actually need to install them. You can easily upload your screenshots via a web page. This makes it platform-agnostic; you can browse, search, and ask about your historical screenshots at any time via a browser. Of course, it doesn't occupy any of your local storage. Last but not least, it's Windows-user friendly.

[3] User-controlled screenshots mean that the embedded chunks are semantically more precise. Beyond the advanced reasoning capabilities of GPT-4, we've integrated some tricks of our own, allowing Trickle to truly comprehend your screenshots, rather than just summarizing the text. Sometimes you might be surprised when Trickle even reasons out essential information that's not present in the image. This also ensures a better experience when you try to recall information.

------------The Story Behind the Product:------------

In 2018, I left my consulting role and dove head-first into the startup world with two co-founders. Our initial venture involved creating a visual recognition model for a vending machine company, marking our first income. Yet, custom builds weren’t sustainable, prompting our first pivot.

By 2019, we were deep into retail tech, winning a demo day and launching a product to automate in-store promotions. The climax seemed to be our partnership with a large multinational, but 2020 and the pandemic shifted landscapes.

We then explored the realm of asynchronous video, building an alternative to Loom. By 2021, we hit 10k users, but profitability remained elusive. As workspaces evolved and people returned to offices, we sensed another opportunity. Our solution? An in-house social platform for teams, named "Trickle". In 2022, after a launch on Product Hunt, even though we garnered attention from people like Ryan Hoover and former Microsoft VP Lu Qi, we faced a stark realization: Daily team updates might not be as 'cool' as anticipated.

Our anxiety steered us off-path, and soon, Trickle became a bulky hybrid, attempting to replace tools like Notion and Slack. Looking back, this detour was a misstep.

The turning point came mid-2023. A series of tepid Product Hunt launches forced introspection. We stripped Trickle down, retaining only its name and began anew. Today, with the original team intact, we're addressing an everyday issue that resonated with all of us: the chaos of managing screenshots.

Reflecting on our journey, it's clear that the essence of the startup spirit lies in adapting, evolving, and pursuing that 'Eureka' moment, no matter how winding the path.




Just a random question, but would most of your service not become obsolete when OpenAI releases their advertised image understanding capabilities for GPT-4?

EDIT: And what about privacy? What if some of my screenshots potentially contain NSFW/NSFWL text/imagery?


In fact, we've built a wealth of application-layer features based on OpenAI's API, many of which I believe OpenAI wouldn't necessarily pursue:

- A screenshot library supporting fast search.

- Ask AI optimized for screenshot contexts.

- A Mac screenshot client tool.

- Our upcoming Mac and iPhone apps, which will likely allow screenshots to be stored on the devices.

- Integrations: for instance, auto-syncing your screenshots and generated digests to Notion; or uploading screenshots from Discord without even having to open Trickle, among others.

- Leveraging the screenshot library to deliver more value, such as collating user feedback screenshots and periodically generating user insights from them.

We're eagerly looking forward to GPT-4's image understanding capabilities being released so we can offer our users an even more imaginative experience.


If your screenshots have any NSFW or NSFWL content, we'd suggest giving them a quick check before uploading. Our platform's main goal is to help you manage and search your screenshots better, but we won't review or differentiate the content in user uploads.


Respect for persistence.

For the screenshot thing, are you using the GPT-4 visual understanding? I assume not since it's only available for a few groups.

Looks like an interesting project.

Are you using BLIP-2? I understand if you don't want to give away secrets. Just thought it couldn't hurt to ask.


Thanks for the support!

We only used the standard GPT-4 API (without visual capabilities).

I think it's okay to have a some discussion here. We employed traditional OCR, GPT-4, and some of our own algorithms to assist GPT-4 in understanding the context and relationship within the image scene (allow me to retain a bit of mystery here).


What about the privacy concerns mentioned above?

Are you doing all this work locally on the device, or are you shipping everything to OpenAI servers, or maybe other servers in some cloud?

If you're doing all processing locally on the device, I might be interested. But I don't want all that data being shipped off to some cloud for remote processing there.


In our current web app, users' screenshots are stored on AWS cloud. Only the essential information processed from these screenshots is sent to the OpenAI API. According to OpenAI API's public commitment, this data will not be used for any training purposes.

For our upcoming Mac client and iPhone app, we're considering storing the screenshots on the device. However, we still need to weigh the pros and cons of cross-device syncing versus local storage. Your feedback is invaluable in guiding our final decisions for the Mac client and iPhone app. Thanks for sharing :)


When you've got a local-only solution, or where local-only is at least available as a configuration option, please let me know.

Thanks!


Absolutely! We'll keep you posted once a local-only option is available. Thanks for your interest!

By the way, are you more excited for the iPhone app or the Mac client?


I don't generally take screenshots on my iPhone, only my Mac. But I'm sure a lot of people would like to have this app on iOS, too.


Thanks for your feedback :)


Great idea and implementation. Is this a competitor to rewind?


While it might seem like we're on a similar tech trajectory as Rewind, we're actually diving into totally different use cases. Rewind is all about enhancing memory. This means when it's recording automatically, users don't really know which parts will be valuable until they think back and search later on.

Trickle, on the other hand, is all about helping users manage those fragmented bits of info via screenshots. So when someone takes a screenshot, it's because they know there's a chance they'll need it somewhere down the line. Hope that clears things up!




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: